In skip-gram training, what is the model's objective?

In skip-gram training, what is the model's objective?

• A. To predict surrounding context words from a center word.
• B. To predict the next sentence given the previous sentence.
• C. To classify documents by topic.
• D. To maximize the distance between word vectors.

Answer: (A)

In skip-gram training the model learns by using a center word to predict its surrounding context words within a fixed window. The objective is to maximize the probability of those context words given the center word across the training corpus, which pushes words that appear in similar contexts to have similar embeddings. In practice this means optimizing a sum of log probabilities p(context_word | center_word) over all position pairs in the text, often approximated with techniques like negative sampling to keep training tractable. This focus on predicting nearby words from a center word is what distinguishes skip-gram from other tasks, such as predicting the next sentence or classifying documents, and it aligns the learned word vectors so that words that co-occur frequently are placed close together. It’s not about maximizing distances between vectors; instead it’s about capturing contextual similarity through prediction.

In skip-gram training the model learns by using a center word to predict its surrounding context words within a fixed window. The objective is to maximize the probability of those context words given the center word across the training corpus, which pushes words that appear in similar contexts to have similar embeddings. In practice this means optimizing a sum of log probabilities p(context_word | center_word) over all position pairs in the text, often approximated with techniques like negative sampling to keep training tractable. This focus on predicting nearby words from a center word is what distinguishes skip-gram from other tasks, such as predicting the next sentence or classifying documents, and it aligns the learned word vectors so that words that co-occur frequently are placed close together. It’s not about maximizing distances between vectors; instead it’s about capturing contextual similarity through prediction.