Training data is the corpus of examples (text, images, code, audio, video) used to train AI models. The quality and scale of training data are two of the three key inputs (alongside model size and compute) that determine final model capability per the empirical scaling laws. High-quality training data is increasingly the constrained resource in AI development as compute scales faster than data quality. It's the input that becomes the output: what the model can do is bounded by what it learned from.
The components of modern AI training data:
Pre-training data (foundation model training):