Practical Day 1 – GRA 4164

<p class="kicker">GRA 4164 · Lecture 5 · Fall 2026</p>

## Practical Day 1

### Leif Anders Thorsrud & Vegard H. Larsen — BI Norwegian Business School

---

## How today works

- Short walkthroughs of the **three demos** — then you code
- We circulate; **bring your assignment problems**
- Goal: leave with the lecture 1–3 toolkit **running on your machine**

---

## Working with code, in 2026

- **Virtual environments** — one per project, always
- **Start small, test often** — run every few lines, not every few hours
- **Use AI assistants** — and **verify** what they give you (lecture 7's rule, early)
- **Read the error message** — debugging is the actual skill

</div>

---

## Today's toolbox

**Classic NLP**   `NLTK` — tokenize, stem, POS-tag

**ML workhorse**   `scikit-learn` — TF-IDF, classifiers

**Topic models**   `gensim` — LDA, word2vec

**Later in the course**   `PyTorch` `spaCy`

</div>

---

## Demo 1: part-of-speech tagging

```python
from nltk import word_tokenize, pos_tag
tokens = word_tokenize("The quick brown fox jumps over the lazy dog.")
pos_tag(tokens)   # [('The','DT'), ('quick','JJ'), ('fox','NN'), ...]
```

- Word classes: **NN** noun · **VB** verb · **JJ** adjective · **DT** determiner
- Use case: filter a corpus to **nouns and adjectives** before topic modelling

---

## Demo 2: TF-IDF in three lines

```python
from sklearn.feature_extraction.text import TfidfVectorizer
tfidf = TfidfVectorizer().fit_transform(docs)
pd.DataFrame(tfidf.toarray(),
             columns=vectorizer.get_feature_names_out())
```

- Lecture 2's math, productionised — inspect the matrix in pandas

---

## Demo 3: LDA on real data

```python
dictionary = corpora.Dictionary(processed_docs)
dictionary.filter_extremes(no_below=15, no_above=0.5)
corpus = [dictionary.doc2bow(d) for d in processed_docs]
lda = gensim.models.LdaModel(corpus, num_topics=10,
                             id2word=dictionary, passes=10)
```

- 2,000 newsgroup posts → **10 topics** in ~a minute
- The full pipeline: clean → tokenize → filter → dictionary → BoW → model

---

## Assignment 1

**Work block: your group, your assignment.** Today's pipeline — preprocess → represent → model — is the skeleton of assignment 1. Get the skeleton running on *your* data today; we circulate.

</div>

---

## In sum

- The lecture 1–3 toolkit **runs on your machine**
- One pipeline everywhere: **clean → represent → model → interpret**
- AI helps you code; **you** answer for the result

</div>

---

## Questions

---

## What's next

- **Lecture 6 (Tue): word embeddings** — words become vectors with meaning
- **Before then:** read **Kozlowski et al. (2019)** — it's discussed in the lecture
- Keep the notebook environment alive — practical day 2 builds on it