Pre-training

The initial, resource-intensive phase of training a large model on a massive, general dataset. This gives the model a broad understanding of language and the world, which can then be adapted through fine-tuning.