King − man + woman, explained

One of the most famous results in word embeddings is a piece of arithmetic: take the vector for king, subtract man, add woman, and the closest word to the answer is queen. It sounds like a magic trick. It is really a statement about directions in space, and once you see that, you can also see where it works and where it fails.

Words as arrows

In an embedding model each word is a list of numbers, here 300 of them, called a vector. You can picture that list as an arrow from the origin (the point where every number is zero) to a point. Adding and subtracting arrows works one number at a time: to compute king − man, subtract the first number of man from the first number of king, then the second, and so on, all the way to 300.

The result of a subtraction is itself an arrow: the direction you would travel to get from man to king. If the model has learned a consistent notion of “royalty”, then the trip from woman to queen should point roughly the same way.

So king − man + woman means: start at king, remove the “man” direction, add the “woman” direction. If the geometry is clean, you land near queen. The same idea can be read the other way around, as “what is to woman as king is to man?”

What the Vector Playground actually does

The Playground has three boxes, a, b and c, and computes a − b + c. Behind the scenes it:

• looks up the full 300-number vector for each of your three words;
• does the arithmetic on all 300 numbers;
• compares the result with every word the model has loaded (10,000 of them) using cosine similarity, which measures how closely two arrows point the same way;
• skips your three input words, and reports the closest remaining words and their similarity.

GloVe and FastText both land on queen. Word2Vec does not, and the reason is mundane: its vocabulary is case-sensitive, and the 10,000 words loaded here include only a capitalised Queen. That vector scores 0.44, below royal.

The fine print

That step “skips your three input words” matters more than it looks. In the GloVe data, the vector closest to king − man + woman is actually king itself (similarity 0.82), with queen second. Subtracting man and adding woman nudges the arrow only slightly, so it stays nearest to where it started. Almost every analogy tool excludes the inputs for this reason. The famous result is real, but it is a little less dramatic than it sounds.

The 3D arrows on the canvas are another simplification: they are a flattened view of 300-dimensional vectors, so angles and lengths on screen are only approximate. The picture is a hint; the answer comes from the full 300-number calculation.

More analogies to try

These all work with GloVe, the Playground’s default model:

• paris − france + italy → rome (capital cities)
• france − paris + tokyo → japan (the same relation, reversed)
• walking − walk + swim → swimming (verb forms)
• bigger − big + small → smaller, with larger a very close second

That last one shows a common failure. smaller and larger appear in nearly identical sentences, so the model keeps them close together. Embeddings are good at “same kind of word” and weaker at “opposite of”.

Reading the result

The similarity the Playground shows is cosine similarity: 1 would be a perfect match in direction. It is not a probability or a confidence score. It only says how closely the result arrow points toward each candidate word. A low number, like Word2Vec’s 0.51, means the calculation landed in a sparse area and even the nearest word is not very near.

You can also skip the calculation and just plot words. The words box takes up to 50 words and draws each one as an arrow, which is a quick way to compare a small set like man woman king queen.

Why it works, and what it reflects

Nobody programmed a “royalty” or “gender” direction into these models. They emerged because the training text uses king and queen in parallel ways, just as it does man and woman. The geometry is a compressed summary of how people write. Why directions in an embedding carry meaning at all is explored in Latent space: the hidden meaning in numbers.

That cuts both ways. Researchers have shown that the same arithmetic surfaces stereotypes from the training text, for example linking certain jobs more strongly with one gender. When an analogy gives a surprising answer, it is worth asking what in the source text might have produced it.