Embedding Explorer
Twenty words in six dimensions, projected to two by PCA. Click a word and compare the neighbours the plot shows with the neighbours the model actually has. They disagree, and the disagreement is not a bug in the plot — it is what projecting six dimensions onto two must do.
The projection distorts high-dimensional geometry. Two components here keep 67.7% of the variance, which means the rest was thrown away to make the plot — and words that differed mostly along the discarded directions land on top of each other. The neighbours the plot shows are not the neighbours the model sees. Every 2-D embedding picture in every slide deck has this property; the only difference here is that the page can count how often it lies.
The projection
Colour is the word's hand-assigned group. Click a word. Solid ring: a neighbour in both spaces. Red dashed: a true neighbour the plot hides. Grey dashed: a neighbour the plot invents.
- farm0.922
- paris0.851
- tokyo0.839
- farm0.999
- whale0.951
- paris0.791
The plot got 1 of 3 neighbours wrong for this word. The red entries are the neighbours the model uses and you would never have guessed from the picture.
Where the projection lies most
Every word, its three nearest neighbours in the full space and in the plot, and how many agree. Worst first.
| Word | Model neighbours (6-D) | Plot neighbours (2-D) | Agree |
|---|---|---|---|
| travel | runfarmparis | server, laptop, python | 0 / 3 |
| cat | dogwhalechicken | dog, eat, run | 1 / 3 |
| whale | dogcatfarm | kitchen, farm, bread | 1 / 3 |
| compile | pythonlaptopserver | python, laptop, run | 2 / 3 |
| dog | catwhalerun | cat, eat, run | 2 / 3 |
| eat | runchickendog | cat, dog, run | 2 / 3 |
| farm | kitchenparistokyo | kitchen, whale, paris | 2 / 3 |
| kitchen | farmparistokyo | farm, whale, paris | 2 / 3 |
| laptop | serverpythoncompile | travel, server, python | 2 / 3 |
| python | laptopcompileserver | compile, laptop, run | 2 / 3 |
| run | eattraveldog | dog, cat, eat | 2 / 3 |
| server | laptopdatacenterpython | travel, laptop, datacenter | 2 / 3 |
| steak | sushiapplebread | apple, chicken, sushi | 2 / 3 |
| apple | breadsushisteak | sushi, steak, bread | 3 / 3 |
| bread | applesushisteak | sushi, apple, steak | 3 / 3 |
| chicken | steaksushiapple | steak, apple, sushi | 3 / 3 |
| datacenter | tokyoserverparis | tokyo, paris, server | 3 / 3 |
| paris | tokyofarmdatacenter | tokyo, datacenter, farm | 3 / 3 |
| sushi | applesteakbread | apple, bread, steak | 3 / 3 |
| tokyo | parisdatacenterfarm | paris, datacenter, farm | 3 / 3 |
Cosine similarity, by hand
Pick two words. The six numbers are the whole model; the cosine is the dot product over the product of the norms.
Try chicken against apple and against cat: it is close to both, for different reasons, along different dimensions. A 2-D plot has to put it in one place.
How to read this page honestly
What the model is, and what it deliberately refuses to be.
- SIMULATEDThe projection is real principal component analysis — the vectors are centred, the 6×6 covariance is formed, the top two eigenvectors are found by power iteration. Nothing about the 2-D layout is placed by hand, and every neighbour list and cosine on this page is computed from the vectors shown.
- SIMPLIFIEDThe twenty vectors are hand-crafted in six named dimensions — animacy, place-ness, edibility, action-ness, technicality, size — so the geometry can be read. A learned embedding has hundreds of unnamed dimensions and no axis means anything on its own; the distortion you see here is *milder* than the one a real 300-D to 2-D plot produces.
- GENERALCosine similarity compares direction and ignores magnitude. That is the right choice when the vectors were trained for it and the wrong one when magnitude carries meaning — some models encode frequency or confidence in the norm.