Cleaning, Labeling & Preparing Data

Raw data is almost never ready to train on directly. It is famously said that data scientists and ML engineers spend a large majority of their time on cleaning and preparation, not on the modeling itself, and that reputation is well earned.

• -Handling missing values means deciding whether to fill them in with a reasonable estimate, discard the incomplete example, or leave a gap the model can learn to handle.
• -Removing duplicates and near-duplicates matters because repeated examples can quietly bias a model toward whatever happens to be over-represented.
• -Fixing inconsistent formats is necessary because the same underlying value written multiple different ways, dates, units, capitalization, needs to be standardized before it can be compared meaningfully.
• -Filtering out low-quality or irrelevant examples matters too. Not every collected example is actually useful, and including junk can hurt more than help.

For supervised learning specifically, someone or something has to attach the correct label to each example. This can mean human annotators reviewing examples one by one, using labels a product already naturally produces (a purchase is itself evidence a recommendation worked), or, as covered earlier, the self-supervised trick of generating labels automatically from the data's own structure.

Especially for classical ML, less so for deep learning which often learns useful representations directly from raw data, practitioners frequently transform raw data into more useful features before training. Turning a raw timestamp into "day of week" and "hour of day" is a classic example, since those derived values are often more directly predictive than the raw timestamp itself.