Where Training Data Comes From

Every model this course has discussed needs data to learn from, and that data has to come from somewhere. Where it comes from shapes everything downstream, including what the model can learn, who it might unfairly favor or disadvantage, and what legal exposure a company might have.

• -Collected or logged data is what your own product already generates: clicks, purchases, support tickets. Often the richest and most relevant source, if you have a product running long enough to accumulate it.
• -Licensed or purchased data is bought from a data vendor, or licensed from a rights holder, common when you need something your own product does not naturally generate.
• -Public or scraped data is gathered from publicly available sources like the web. It is the primary source for the massive datasets behind large language models, and a genuinely contested area regarding copyright and consent.
• -User-generated or crowdsourced data is gathered directly for the purpose of training, often through a labeling platform or a deliberately designed data-collection product.
• -Synthetic data is generated artificially, sometimes by another model, to supplement or entirely stand in for real data. It is increasingly common but requires care that it actually reflects real-world patterns.

A recurring failure pattern worth internalizing early: a model trained on data from one population, say one country, one demographic, one time period, often performs noticeably worse when deployed against a different population. This is not a hypothetical edge case. It is one of the most common reasons a model that looked great in testing underperforms in the real world.