What constitutes negative data in this training approach?

What constitutes negative data in this training approach?

• A. Pairs drawn from observed co-occurrences.
• B. Pairs drawn from random word pairs.
• C. Pairs involving the same word.
• D. Pairs drawn from non-co-occurring contexts.

Answer: (D)

Negative data are the examples the model should learn to ignore or assign low probability to. In this training approach, they come from word pairs that do not co-occur in the observed corpus. They provide a contrast to the true co-occurring pairs (positive data), helping the model learn to distinguish real associations from random or unrelated pairs. Think of it as teaching what isn’t a meaningful connection: by exposing the model to many non-co-occurring pairs, it learns to push apart the embeddings of unrelated words and avoid overestimating associations. Pairs drawn from observed co-occurrences are positive data, not negative. Pairs from random word pairs could include some actual co-occurrences by chance, so they’re not guaranteed to be non-co-occurring. Pairs involving the same word aren’t inherently negative—they may or may not reflect a meaningful context.

Negative data are the examples the model should learn to ignore or assign low probability to. In this training approach, they come from word pairs that do not co-occur in the observed corpus. They provide a contrast to the true co-occurring pairs (positive data), helping the model learn to distinguish real associations from random or unrelated pairs.

Think of it as teaching what isn’t a meaningful connection: by exposing the model to many non-co-occurring pairs, it learns to push apart the embeddings of unrelated words and avoid overestimating associations.

Pairs drawn from observed co-occurrences are positive data, not negative. Pairs from random word pairs could include some actual co-occurrences by chance, so they’re not guaranteed to be non-co-occurring. Pairs involving the same word aren’t inherently negative—they may or may not reflect a meaningful context.