Markov models primarily capture which type of information in language models?

Markov models primarily capture which type of information in language models?

• A. Global semantic relationships between words
• B. Local word sequence probabilities
• C. Syntactic tree structures
• D. Topic distributions over documents

Answer: (B)

Markov models are built to capture how likely a word is to follow a short, recent context, i.e., local word sequence probabilities. They rely on the Markov assumption, estimating P(w_i | w_{i-1}, ..., w_{i-n+1}) for a small n, so the model focuses on immediate word-to-word or short-phrase dependencies. This makes them effective at modeling the flow of text in small windows, but they don’t encode broader semantic relationships across an entire sentence or document, nor do they produce explicit syntactic structures like parse trees, and they aren’t designed to infer topic distributions over documents. For these reasons, the information most accurately described by Markov models is the local sequence probabilities of words.

Markov models are built to capture how likely a word is to follow a short, recent context, i.e., local word sequence probabilities. They rely on the Markov assumption, estimating P(w_i | w_{i-1}, ..., w_{i-n+1}) for a small n, so the model focuses on immediate word-to-word or short-phrase dependencies. This makes them effective at modeling the flow of text in small windows, but they don’t encode broader semantic relationships across an entire sentence or document, nor do they produce explicit syntactic structures like parse trees, and they aren’t designed to infer topic distributions over documents. For these reasons, the information most accurately described by Markov models is the local sequence probabilities of words.