Embeddings
Discrete entities as dense vectors — words, users, products, documents — cosine similarity, and why a 2D projection distorts the geometry it claims to show.
A discrete entity — word, user, product, document — becomes a dense vector that is a parameter of some model, learned on a proxy task. The geometry encodes what that task rewarded, not "meaning".
Lookup tables as parameters, a contrastive signal from observed pairs against sampled negatives, and the consequence: rare entities get noise, and the table is part of the model artifact and must be versioned with it.
The dot product divided by the norms: the angle between two vectors, ignoring their length. Right when magnitude is noise, wrong when magnitude is signal — and at scale, nearest neighbours are an index problem, not a formula.
A 2D plot of high-dimensional vectors is a lossy projection. PCA keeps variance, not neighbourhoods; t-SNE and UMAP keep local structure and invent global structure. The clusters, distances and neighbours in the picture are not the ones the model uses.
Retraining an embedding model produces a new coordinate system. Vectors stored from the old model are incompatible with it — a version mismatch, not a quality problem — and the vocabulary and the entities drift underneath as well.