Introduction to text as data
What text data is, why it matters for business, and the basic steps for preparing text for analysis
Open slidesOverview
Most of the information firms sit on is text, not tables: news, customer reviews, filings, e-mails, social media. This first lecture sets up the whole course — what it means to treat text as data, what becomes possible when you do (predicting markets from news, listening to customers at scale, mapping policy positions), and why it is hard: scale, noise, ambiguity, and ethics.
We also walk through how the course works: the 12-lecture arc from counting words to large language models, the four group assignments, and the exam.
Topics
- Text as a data source: examples from news, social media, reviews, and filings
- The text-as-data workflow: collect → preprocess → represent → model → interpret
- Why it’s hard: scale, noise, ambiguity, and ethics
- Course practicalities: assignments, exam, tools, and reading
Readings
- J&M chapter 2
Materials
- Slides: use the Open slides button above
- Notebook: Working_with_text_data.ipynb — Python warm-up from strings and regex through pandas, NLTK, scikit-learn, and spaCy. No external data needed.
Before lecture 2
- Form a group (1–3 students) via the group allocation on Itslearning
- Install Python and Jupyter — see the Tools page
- Run the notebook above top to bottom; bring questions on Tuesday