Embeddings

When a neural network is trained on a particular task called language modeling —predicting the next word in a sequence —the embeddings it learns are anything but arbitrary. 

Like iron filings lining up under a magnetic field, the values become set in such a way that words with similar associations have mathematically similar embeddings. For example, the embeddings for “dog” and “cat” will be more similar than those for “dog” and “chair.”

This phenomenon can make embeddings seem mysterious, even magical: a neural network somehow transmuting raw numbers into linguistic meaning, “like spinning straw into gold,” [Ellie] Pavlick said. 

Famous examples of “word arithmetic” —“king” minus “man” plus “woman” roughly equals “queen” —have only enhanced the aura around embeddings. They seem to act as a rich, flexible repository of what an LLM knows.