During training, how do language models learn?

During training, how do language models learn?

• A. Predicting the next word, showing billions of text examples, adjusting weights.
• B. Classifying sentences into topics.
• C. Translating text from one language to another.
• D. Compressing model parameters to save memory.

Answer: (A)

Training a language model centers on predicting the next word in a sequence given the preceding context. By exposing the model to billions of text examples, it learns patterns, grammar, and word usage, and it refines its internal weights through backpropagation to minimize prediction error. Each step compares the model’s guess for the next token to the actual token, computes a loss, and updates the weights accordingly. Over many examples and iterations, the model develops representations that capture how language tends to unfold, enabling coherent and contextually appropriate text generation. Other tasks listed—classifying sentences into topics, translating text, or compressing model parameters—are different objectives or techniques. Classifying topics is a supervised discrimination task, translation is a separate sequence-to-sequence objective, and compression focuses on reducing memory use rather than learning from data.

Training a language model centers on predicting the next word in a sequence given the preceding context. By exposing the model to billions of text examples, it learns patterns, grammar, and word usage, and it refines its internal weights through backpropagation to minimize prediction error. Each step compares the model’s guess for the next token to the actual token, computes a loss, and updates the weights accordingly. Over many examples and iterations, the model develops representations that capture how language tends to unfold, enabling coherent and contextually appropriate text generation.

Other tasks listed—classifying sentences into topics, translating text, or compressing model parameters—are different objectives or techniques. Classifying topics is a supervised discrimination task, translation is a separate sequence-to-sequence objective, and compression focuses on reducing memory use rather than learning from data.