What is a shared characteristic of large language models and word embedding models?

What is a shared characteristic of large language models and word embedding models?

• A. They both compress text into discrete symbols.
• B. They both convert text into high-dimensional numerical vectors.
• C. They both require supervised labels.
• D. They both generate only binary outputs.

Answer: (B)

Both large language models and word embedding models rely on distributed representations that turn text into numerical vectors. Each token is mapped into a high-dimensional vector, and through training these vectors capture semantic and contextual information so the model can compare words and understand relationships. This vector-based view lets the system measure similarity, capture nuances of meaning, and feed these representations into deeper layers that model syntax and context over sequences. The key takeaway is that turning text into high-dimensional numerical vectors is the shared hallmark of both approaches. Discrete symbol compression isn’t the hallmark here, since these models operate in continuous vector spaces rather than relying on fixed discrete symbols. Supervised labeling isn’t required as a universal rule—embeddings can be learned with unsupervised objectives, and language models often start with unsupervised pretraining. And outputs aren’t limited to binary results; they generate probabilities over a vocabulary, enabling a range of possible next tokens rather than just true/false.

Both large language models and word embedding models rely on distributed representations that turn text into numerical vectors. Each token is mapped into a high-dimensional vector, and through training these vectors capture semantic and contextual information so the model can compare words and understand relationships. This vector-based view lets the system measure similarity, capture nuances of meaning, and feed these representations into deeper layers that model syntax and context over sequences. The key takeaway is that turning text into high-dimensional numerical vectors is the shared hallmark of both approaches.

Discrete symbol compression isn’t the hallmark here, since these models operate in continuous vector spaces rather than relying on fixed discrete symbols. Supervised labeling isn’t required as a universal rule—embeddings can be learned with unsupervised objectives, and language models often start with unsupervised pretraining. And outputs aren’t limited to binary results; they generate probabilities over a vocabulary, enabling a range of possible next tokens rather than just true/false.