Encoder / Decoder Families
Encoder-only models turn text into a representation and are what you want for classification and embeddings; decoder-only models generate the next token; encoder–decoder models read one sequence and write another. Pick by what the output must be, not by which is newest.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
Encoder-only, decoder-only and encoder–decoder transformers exist side by side. What does each produce, and which one gives you the embeddings a retrieval system needs?
A knowledge-base team wants three things: tag every article with topics, let users search semantically, and translate the whole base into two more languages. The engineer proposing the plan has one large decoder-only model in mind for all three because "it can do anything", and the infrastructure team is asking why search latency is being quoted in seconds.
One generative model for everything. Prompt it to emit tags, prompt it to score each article against the query, prompt it to translate. One model to serve, one API to learn.
Search through a generative model means running the model per query per candidate — or asking it to rank a list — at generation speed. The latency is seconds and the cost is per token; the budget was milliseconds. Offline relevance looked fine because nobody timed it.
- Search through a generative model means running the model per query per candidate — or asking it to rank a list — at generation speed. The latency is seconds and the cost is per token; the budget was milliseconds. Offline relevance looked fine because nobody timed it.
- Tags emitted as free text are strings, not classes: the model writes "Data privacy" and "data-privacy" and "GDPR" for the same section, and the downstream join to the section list silently drops a fraction of every batch. The offline check compared prose to prose and passed (Structured Outputs on the Agentic side is the mitigation for this, at a cost).
- The same model is being asked to be three artifacts. A change to improve translation quality changes tagging behaviour, and there is no evaluation that says so until editors complain.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- Tagging: a multi-label classification per article, labelled by editors; the decision is which sections an article appears in.
- Search: for a query, rank articles by relevance; the label is a click or an editor-judged relevance grade; the decision is the top ten shown.
- Translation: a target-language article for each source article; judged by bilingual reviewers on a sample.
- Articles are a few hundred to a few thousand tokens each, tens of thousands of them, with editor tags for most and relevance judgements for a small query set.
- Search traffic is thousands of queries per minute with a latency budget in the low hundreds of milliseconds (Latency Budgets: Spending 200 Milliseconds on Purpose).
- Translation is a batch job with no latency budget and a quality bar set by reviewers.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- An encoder-only model applies bidirectional attention — every token sees every other — and outputs one vector per input token. It produces representations, not text: a pooled or
[CLS]vector for classification, or a normalised vector for similarity. It is trained by predicting masked tokens, which forces each position to summarise its context. This is the family that produces the Embeddings a retrieval index stores. - A decoder-only model applies causal attention — each token sees only what came before — and is trained to predict the next token. Its native output is a probability over the vocabulary, sampled one token at a time; its representation of the input exists but is shaped for generation, not for symmetric similarity. It is what you want when the output is new text.
- An encoder–decoder model encodes the input bidirectionally, then a decoder generates the output while cross-attending to the encoded input. It is the natural fit for sequence-to-sequence tasks — translation, summarisation with a strong grounding requirement — where the whole input is available before any output is written.
Three masks, three outputs
The families differ in one line of the attention code — the mask — and in the training objective, and those two choices decide what the model is good at producing. Bidirectional attention with a masked-token objective makes every position a summary of its whole context, which is what a representation is. Causal attention with a next-token objective makes the last position a predictor of what comes next, which is what a generator is.
Encoder–decoder keeps both: a bidirectional encoder over the input, a causal decoder over the output, and cross-attention from every decoder position to every encoder position. The decoder never has to hold the input in its own state because it can look at the encoding directly.
| Family | Attention | Pretraining objective | Native output | Good at | Serving shape |
|---|---|---|---|---|---|
| Encoder-only | Bidirectional | Masked-token prediction | One vector per token; pooled vector | Classification, tagging, embeddings for retrieval, similarity | One forward pass; milliseconds; batchable |
| Decoder-only | Causal | Next-token prediction | Distribution over next token | Open-ended generation, instruction following, chat | One pass per generated token; seconds; KV-cache bound |
| Encoder–decoder | Bidirectional in, causal out, cross-attention | Denoising / span corruption, seq-to-seq | Output sequence grounded in the encoded input | Translation, grounded summarisation, structured transduction | Encode once, then per-token decode; batch-friendly |
Which family makes the retrieval embeddings
Semantic search needs a vector per article that can be computed once and stored, and a vector per query computed at request time, with a similarity between them that reflects relevance. That is an encoder trained with a similarity objective: pairs of related texts pulled together, unrelated pushed apart, so that a dot product means something (Embedding Training).
A decoder's hidden states exist and can be pooled, but they were shaped to predict the next token, not to be symmetric across a query and a document. Without further training with a contrastive objective, the neighbourhoods they produce are about surface form and boilerplate. The retrieval index, the chunking and the query pipeline built on top belong to Agentic Engineering; which model produces the vectors is this domain's call (Vector Storage, RAG Overview).
Every vector in the index and every query vector were produced by the same encoder weights and the same preprocessing, so their dot products are comparable.
holds when The encoder version is pinned in the artifact, the index records which version built it, and a re-index accompanies every encoder change (What a Model Artifact Contains).
breaks when The query service picks up a newer encoder from the registry; a re-index runs partially; tokeniser or normalisation differs between the batch embedding job and the query path (Train / Serve Skew).
respond Refuse to serve on a mismatch and fall back to lexical search; re-index fully before promoting the encoder.
For each query, prompt the decoder with the query and each of the top hundred candidates and ask for a relevance score. A hundred generative passes per query; seconds of latency; cost per token.
Embed articles once; embed the query in one encoder pass; nearest-neighbour lookup; re-rank the top twenty with a small cross-encoder. Tens of milliseconds; cost per query fixed.
The encoder's output is a vector that can be pre-computed and compared with a dot product, so nearly all the work is done before the query arrives. Generation cannot be pre-computed because it depends on the query, and it pays per token.
Three tasks, three artifacts
The plan that survives is three models: an encoder fine-tuned for tagging with a per-class head, an embedding encoder plus a re-ranker for search, and a sequence-to-sequence model for the translation batch. Each has its own evaluation set, its own latency profile and its own registry entry, and a change to one cannot regress another (The Model Registry).
What the single-model plan offered was one API to learn. What it cost was untyped outputs, generation latency where milliseconds were required, and an evaluation that could not tell the three jobs apart. The families are not a history lesson; they are the reason the outputs have the shapes they have.
What must the model produce, and under what latency?
when The output is a class, a score, or a vector for similarity, and it must be fast and batchable.
cost Cannot generate text; needs a task head and labelled data to fine-tune; embeddings need a similarity objective to be useful for search.
when The output is new text conditioned on a prompt, and generation latency and per-token cost are acceptable.
cost Output is untyped text unless constrained; per-token cost and latency; representations not shaped for retrieval without extra training.
when The whole input is known up front and the output is a transformed sequence — translation, grounded summarisation.
cost Two stacks to serve; fewer large pretrained options today than decoder-only; still generation-speed on the output side.
How to build it
Most important first.
- Choose by output type. A class or a vector: encoder. New text conditioned on a prompt: decoder. A sequence transformed into another sequence with the full input known up front: encoder–decoder, or a decoder-only model where one is much better pretrained.
- For search, embed articles once with an encoder into a vector index and embed the query at request time; the model runs once per query, in milliseconds, and a small cross-encoder re-ranks the top few (Embeddings, Cosine Similarity). The retrieval system built on top is Agentic Engineering's subject (Dense, Sparse & Hybrid Retrieval).
- For tagging, fine-tune an encoder with a multi-label head on the editor tags, so the output is a probability per known class, thresholded per class (Fine-Tuning, Threshold Selection).
- For translation, use an encoder–decoder or a decoder-only model pretrained for it, as a batch job with reviewer sampling; it has no latency budget and the highest quality bar, and it should be its own artifact (Batch Inference).
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Search: recall of relevant articles in the top ten and p99 query latency — both, because the family choice trades one for the other (Ranking).
- Tagging: per-class precision and recall against editor tags on held-out articles, not "does the text look right".
- Do not measure the plan by "the model can do all three in a demo". It can. The numbers that decide are latency per query and the join rate of tags to sections, neither of which the demo shows.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The output type each task needs — class, vector, text — does not change; a product request to "explain the tag" turns a classification into generation and changes the family.
- The encoder that embedded the index is the encoder that embeds the queries, byte for byte; an upgraded query encoder against a stale index is a broken search that returns confident results (Embedding Drift).
- The latency and cost profile of each family holds at production traffic — an encoder pass per query in milliseconds, generation in seconds — and traffic growth does not push the re-ranker over the budget.
- Offline: evaluate each task with its own metric on its own held-out set; a shared demo prompt is not an evaluation of three systems.
- Online: log the query encoder version alongside the index version on every search, and alert on any mismatch; monitor tag-to-section join rate per batch.
- Over time: re-judge a fixed query set quarterly and compare recall at ten; embeddings that were good drift as the article base changes even if the encoder does not (Embedding Drift).
What can go wrong
- The encoder used for search was pretrained on general web text and embeds domain jargon poorly; two articles about different products are neighbours because they share boilerplate (Embedding Training).
- The re-ranker improves relevance and adds forty milliseconds per query; at peak traffic the budget is blown by the improvement.
- The translation model is updated and its output length distribution changes; a downstream layout step that assumed roughly equal length now truncates.
- Three artifacts instead of one means three registries entries, three evaluation sets and three things to monitor; it also means a translation change cannot break search.
- An encoder gives fast, cheap, typed outputs and cannot explain itself in prose; a decoder can, at generation cost, and its prose is not an explanation of the encoder's decision.
- A cross-encoder re-ranker is more accurate than bi-encoder similarity and cannot be pre-computed, so it is applied only to a short list — accuracy on the top few, bought with latency.
- "The big generative model can do classification, so we do not need an encoder." It can produce a label as text. It cannot produce a calibrated probability per known class at encoder speed, and the label strings will not join cleanly. Different output, different tool.
- "Decoder-only won, the other families are obsolete." Decoder-only won for generation and for the models a team can most easily obtain. The embeddings behind nearly every retrieval index are still produced by encoders, and sequence-to-sequence tasks still run on encoder–decoders.
- "Embeddings from any model are fine for search." A decoder's hidden state is trained to predict the next token, not to place similar documents near each other. Retrieval-quality embeddings come from encoders trained with a similarity objective.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThe mapping from output type to family — vector or class from an encoder, text from a decoder, sequence-to-sequence from an encoder–decoder — follows from the attention mask and the training objective, and holds across domains and model sizes.
- CONTESTEDA serious position holds that decoder-only models, trained at scale and adapted with a contrastive objective, now produce embeddings as good as or better than encoder-only models, and that the family distinction is collapsing into "one architecture, several objectives". The leaderboards partly support this; what has not collapsed is the cost — an encoder pass is cheaper than a large decoder pass — and for most teams a dedicated embedding model remains the pragmatic choice.
- SCALE-SPECIFICAt a few hundred articles and a few queries a day, scoring every candidate with a generative model is affordable and the family question barely matters. It becomes decisive at thousands of queries a minute with a millisecond budget, which is where the retrieval architecture is forced.
Where the depth lives
This domain teaches the model and hands the rest off by name.