DSLtypical

Should I Build a DSL?

Almost always no. Four questions decide it — is the domain stable, do the readers genuinely need non-host syntax, is the tooling budgeted, and could a library do it — and a yes needs all four. This lesson is the decision, stated as a decision.

The question

We keep saying this would be cleaner as a small language. Should we actually build one?

SourceLexingTokensParsingASTSemanticsTypedIROptimizeCodegenMachine codeLinkExecute
What the program is here

Not a program at this point but a proposal: a claim that a set of domain rules would be better expressed as programs in a notation that does not exist yet. The representation being chosen is the one the domain logic will live in for years — host code, structured data, an internal notation or an external language — and the decision is which of those four the logic should *be*, given who reads it and who maintains the tools for it.

What this phase may assume or do

A yes is defensible only under all four preconditions together: the domain's concepts have been stable long enough that the notation will still be right in two years; the people who will read these programs are genuinely served by non-host syntax; the tooling — diagnostics with positions, a formatter, editor support, versioning, a migration path — is budgeted and owned by a named person; and no library or schema-validated data format solves the problem. Any one of those failing makes the answer no, and the fourth is the one people skip.

Key points

  • The default answer is no, and stating it first is part of the method — the benefits are vivid and the costs are diffuse.
  • Four questions must all pass: stable domain, readers who need non-host syntax, funded tooling ownership, and no cheaper alternative.
  • The strongest argument for yes is an audience that is not and cannot be host-language programmers.
  • The second strongest is a restriction that buys a guarantee a library in a general-purpose host cannot provide.
  • Syntax is the hardest thing to change later, so an unstable domain is disqualifying on its own.
  • The decision is a ladder — name the concepts, then data plus schema, then internal DSL, then external — and you stop at the first rung that works.
  • A yes commits you to diagnostics, recovery, a formatter, editor support, docs, versioning, migration and a debugging story, indefinitely.
  • It also commits you to refusing reasonable feature requests forever, because the restrictions were the point.

Start from no, and mean it

The default is no, and it should be stated first rather than reached reluctantly at the end. A language is not a feature; it is a product with an indefinite support obligation. The parser is a weekend. Diagnostics that name a line and a column, error recovery so one mistake does not hide the rest, a formatter, syntax highlighting, completion, a debugging story, documentation, a versioning policy and a migration path for existing programs are the actual project, and they never finish.

The reason to start from no is not pessimism. It is that the arguments for a DSL are vivid — here is the ugly code, here is the beautiful notation — and the arguments against are diffuse, arriving over the following three years in the form of a new engineer who cannot change a rule and an error message that says "unexpected token". A decision procedure that weighs a vivid benefit against a diffuse cost without a stated default will say yes almost every time.

The second reason is that most of the value people expect from a DSL comes from naming the domain concepts explicitly, and that is available from a library at a fraction of the cost. If the current code is unreadable because it does not name what it is doing, a language will not fix that; it will express the same confusion in a syntax nobody else knows.

The four questions

Each question below has a way of answering it that does not rely on how you feel about the idea. Use those; the questions are easy to answer optimistically otherwise.

Is the domain stable? Look at the last year of commits to the code this would replace. If the *concepts* changed — not just the values, but what things are called and how they relate — the domain is still moving, and freezing it into a syntax guarantees a migration. Syntax is the hardest thing to change once programs exist in it, because every change is a change to somebody else's files.

Who reads these programs, and does syntax help them? Not who writes them — who reads them during an incident, six months from now. If the answer is "the same engineers who work in the host language", the notation gain is small and the tooling loss is large. If it is analysts, operators or domain experts who will not install a toolchain, that is the strongest single argument available for a DSL, and it is the one that justifies most of the successful ones.

Is the tooling budgeted, and who owns it? Name the person. Put the tooling in the same plan as the language, not in a follow-up ticket. If it cannot be staffed, the answer is no regardless of how good the design is, because an unowned language becomes infrastructure everyone depends on and nobody can change — the most expensive failure available here.

Could a library or a data format do it? Write the three hardest real examples as a library API and as schema-validated data. If the data version needs no conditional, no loop, no name binding and no reuse, it is data and it should stay data. If the library version reads acceptably to someone who did not design it, that is the answer.

The four questions, how to answer them, and what a failure meanstypical
QuestionHow to answer it honestlyIf the answer is bad
Is the domain stable?Read a year of commits to the code this replaces; look for concept changes, not value changesNo. Wait a year and keep the flexibility to refactor with ordinary tools
Who reads it, and does syntax help?Give a draft to one of the actual readers with no explanation and watchNo, if they are host-language programmers. A library names the concepts for a fraction of the cost
Is the tooling owned?Name the person and put the tooling in the same plan as the languageNo. An unowned language stops at "unexpected token" and adoption stops at its author
Could a library or schema do it?Write the three hardest examples both ways and compare as a reader, not as the designerNo. Ship the library, and revisit when it demonstrably cannot express something real
All four passStable domain, non-host readers, funded owner, no cheaper optionYes — and prefer internal first if the host syntax can carry the notation at all

What changes a no into a yes

The single strongest signal is an audience that is not the host language's programmers and cannot be. SQL exists because analysts needed to ask questions of data without writing programs. HCL exists because the people declaring infrastructure are not necessarily application developers. Shader languages exist because the hardware model is not the CPU's. In each case, "just use a library" was not available, because the readers were not in a position to use the host language at all.

The second is when the restriction buys a genuine guarantee. If making the language unable to loop means the configuration cannot hang, or making it unable to do I/O means a build is reproducible, or making the case set closed means the compiler can prove completeness over the domain, then the language is doing something a library in a general-purpose host cannot: it is *removing* capabilities, and a library cannot remove what the host provides.

The third is when the same specification must drive several outputs. One schema producing validation, serialisation, documentation and a client, from a single artifact that can be diffed and reviewed. That requires the program to exist as an inspectable artifact, and it is a real reason.

And a genuine fourth, less often stated: when the domain logic must be authored, reviewed and versioned independently of the application that runs it. A rules file that domain experts change and release on their own cadence is a different artifact from a code change, and making it a language is the honest expression of that.

  • The readers are not host-language programmers and cannot be — the strongest single argument.
  • The restriction buys a guarantee: termination, reproducibility, exhaustiveness over the domain, an optimizable plan.
  • One artifact must drive several outputs, so the program has to be inspectable rather than executed.
  • The domain logic has its own authors, its own review process and its own release cadence.
  • A notation already exists in the domain and the host language visibly fights it.

The graduated answer

typicalThe ladder describes how successful DSLs in industry generally arrived: as an internal API or a data format first, extracted once the notation stabilised. GraphQL, Terraform HCL and several build languages have some version of this history. It is not a rule — a few important DSLs were designed as languages from the start by people who already knew the domain cold, and SQL is the obvious example. The heuristic is about de-risking a decision under uncertainty, not about how good designs are possible.

The question is not binary, and treating it as binary is how teams end up at the expensive end. There is a ladder, and the honest recommendation is to climb it one rung at a time and stop as soon as the problem is solved.

Rung one: name the concepts in ordinary host code. Most "we need a DSL" conversations are actually about code that does not say what it is doing, and this rung solves a surprising number of them at zero cost.

Rung two: a schema-validated data format. If the programs are declarations with no control flow, this is the right size — the tooling comes from the schema ecosystem and the restriction is enforced by the format itself.

Rung three: an internal DSL. Domain notation, the host toolchain, and a wrong design decision is a deprecation rather than a migration.

Rung four: an external DSL with its tooling budgeted. This is where SQL, HCL and GraphQL live, and it is a real destination — just not the first one, and not one to arrive at by accident.

The trap is skipping rungs because rung four is the interesting engineering. It is, and that is exactly why the decision needs a stated default and a written-down set of criteria rather than an argument in a design meeting.

What a yes commits you to

Saying yes is committing to a list, and the list should be in the proposal rather than discovered later. Diagnostics with a line, a column and a suggestion. Error recovery, so a file with two mistakes reports two. A formatter, because otherwise every review argues about layout. Editor support, which in practice means a language server. Documentation with examples. A versioning policy, including what happens to programs written against version one. A migration tool, or an explicit promise never to make a breaking change. And a debugging story, which is the item most often missing entirely — when a program in your language does the wrong thing, how does a user find out why?

It also commits you to saying no repeatedly. Every user will eventually ask for a conditional, then a loop, then a function, then a module system, and each request will be reasonable and supported by a real use case. Holding the restriction is the job, because the restrictions were the reason for the language. A DSL that grants every request becomes a general-purpose language with a worse debugger, which is [[configuration-languages]] in one sentence.

That is the honest bill. It is worth paying occasionally. It is not worth paying because the current code is ugly.

How it works

The steps, in the order the compiler takes them.

  • Write down the three hardest real examples the notation would have to express — real ones from the existing system, not illustrative ones.
  • Express those three as host-language code with well-named concepts, and as schema-validated data. Read both as someone who did not write them.
  • Read a year of history on the code this would replace, and classify each change as a value change or a concept change.
  • Identify the actual readers by name and role, and give one of them a draft with no explanation.
  • Name the tooling owner and put the tooling items in the same plan as the language, with the same delivery date.
  • If all four pass, prefer internal first where the host syntax can carry the notation, and treat the external version as an extraction once the notation stops changing.
  • Write down the restrictions the language will keep and the reason each one exists, so future feature requests can be evaluated against a stated position rather than relitigated.

How it breaks

What the engineer observes when it goes wrong — not what goes wrong internally.

  • The language is built, ships without diagnostics, and adoption stops at its author — the most common outcome by a wide margin.
  • The domain changes six months in, and the fix is either a breaking syntax change or a permanent wart; both are more expensive than the refactor a library would have needed.
  • Tooling is promised as a follow-up and never scheduled, so error messages stay at "syntax error at line 12" for the life of the system.
  • The author leaves and the language becomes load-bearing infrastructure that nobody is willing to touch, so changes are made by editing generated output.
  • Users ask for a conditional, then a loop, then functions, and the language becomes a general-purpose one with a hand-written parser and no debugger.
  • A new engineer cannot make a one-line domain change because the notation is undocumented, so the change takes a week and involves the original author.
  • The DSL solves a readability problem that a library would have solved, and the team now maintains a language to get a benefit they could have had for free.

When it helps

  • Before any code is written, as the checklist that makes the argument explicit instead of a matter of enthusiasm.
  • In a design review, where "which of the four fails?" is a more productive question than "do we like this?".
  • When someone proposes adding a conditional to an existing configuration format, since the same criteria apply to the increment.
  • When inheriting a DSL and deciding whether to keep investing in it, where the same four questions apply retrospectively.

When it hurts

  • As a rule applied without judgement, which would have rejected SQL, HCL and every shader language at their inception.
  • When it becomes an argument against naming domain concepts at all, which is the cheap rung and almost always worth doing.
  • When "no" is used to avoid the harder conversation about why the current code is unreadable.
  • When applied to a decision that is already made and shipped, where the useful question becomes how to fund the tooling rather than whether to have built it.

What it costs

Every one of these is paid by something.

  • Saying no buys the host's entire ecosystem and a change cost proportional to the code, and pays in verbosity: domain code reads as general-purpose code and domain errors stay run-time errors.
  • Saying yes buys notation, a checkable artifact and restrictions that can be guaranteed, and pays a permanent tooling obligation plus a second toolchain in the build with its own CI and its own failure modes.
  • A stated default of no buys protection against a systematic bias toward the interesting engineering, and pays by occasionally rejecting a good idea that would have needed a champion to overcome the default.
  • Climbing the ladder one rung at a time buys de-risking and a chance to discover the notation before freezing it, and pays in rework: the internal API becomes a public interface, and extracting an external syntax later means maintaining two front ends.

What else you could do

What a different compiler or language does instead, and when that is better.

  • Name the domain concepts in ordinary host code. Free, immediate, and it resolves a surprising share of the cases that present as "we need a DSL".
  • A schema-validated data format, which is the right size whenever the programs are declarations without control flow — see [[configuration-languages]].
  • An internal DSL, which gets domain notation while inheriting the host toolchain and keeps design mistakes cheap — see [[internal-vs-external-dsl]].
  • Buy rather than build: an existing DSL that nearly fits, with its tooling and its community, usually beats a perfect one with neither. CEL, Rego, Starlark, Jsonnet and Lua all exist for exactly this reason.

See it for yourself

The flag, dump or tool that shows you this directly.

  • Run the decision tree at /compilers/dsl — the four questions with the how-to-answer guidance attached to each, ending in a verdict with its own tradeoffs and failure modes.
  • Read a year of git log on the code the DSL would replace and classify changes as value or concept. Concept churn is the disqualifying signal.
  • Write the three hardest examples three ways — library, data plus schema, proposed notation — and put them in front of an actual reader without explanation.
  • Look at any DSL already in your codebase and ask the four questions retrospectively. The one that fails is usually the one causing the current pain.
  • Check the proposal for the tooling list and a named owner. Its absence is the single most reliable predictor of the outcome.

Plausible wrong readings

Stated the way a confident engineer states them.

  • "The default is no, so DSLs are a bad idea." SQL, regular expressions and every shader language passed this bar. The default exists because most proposals do not, not because none do.
  • "We are only building a small language, so the tooling cost does not apply." Users of a small language expect the same diagnostics and editor support as users of a large one; the size of the grammar does not shrink the bill.
  • "We will add the tooling once people use it." Adoption is bounded by the tooling, so this order does not converge.
  • "A library cannot express this." Usually it can, less prettily. The cases where it genuinely cannot are the ones where the DSL removes a capability the host provides, which is worth naming explicitly.
  • "The domain is stable because it has not changed recently." Check the concepts, not the values. Configuration keys changing is noise; what a rule *is* changing is the signal.

Misconceptions

The claim, and what is actually true.

Building a DSL is a way to make the code more readable.
Naming the domain concepts is what makes code readable, and that is available from a library. A DSL buys notation and analysis and costs a toolchain.
The hard part is designing the syntax.
The syntax is the fun part and the smallest part. Diagnostics, editor support and migration are the project.
If we regret it we can remove it later.
Once programs exist in a notation, removing it is a migration of somebody else's files, which is why syntax is the least reversible decision in the whole proposal.

Go deeper

The same idea at increasing depth. Stop wherever it stops being useful.

overview

Building a small language for your domain sounds appealing and is usually the wrong call. Ask four things: has the domain stopped changing, do the people who will read these programs actually need a special syntax, is someone funded to build and maintain the tooling, and could a library or a validated config file do the job. You need yes to all four. Most proposals fail on the third or the fourth.

practical

Make the argument explicit. Write the three hardest real examples in each candidate form — plain host code with good names, data with a schema, the proposed notation — and show them to somebody who will have to read them at three in the morning. Then find the tooling owner's name. If the proposal does not contain diagnostics, a formatter, editor support and a migration story with the same delivery date as the parser, it is not a plan, and the outcome is already known.

advanced

The interesting thing about this decision is that it is genuinely asymmetric in a way most architecture decisions are not. A library that turns out to be the wrong abstraction is refactored with the host's tools, and the cost is proportional to the code you own. A syntax that turns out to be wrong is a migration of files other people wrote, and the cost is proportional to adoption — so success makes the mistake more expensive, which is a rare and unpleasant property. That asymmetry, rather than any estimate of implementation effort, is the real reason for a stated default of no: it is not that DSLs are hard to build, it is that the cost of being wrong grows with how well it goes. The correct response is not to avoid languages but to defer the irreversible part — the syntax — until the notation has been validated somewhere cheap, which is exactly what the ladder in this lesson does.

How much this depends on

Nothing in this domain is true of every compiler. These say how much.

typicalThe four criteria describe how DSL decisions generally turn out in industry rather than a law. There are successful DSLs that violated one of them — designed for a domain that was still moving, or shipped with poor tooling and carried by a compelling enough use case. The criteria are a way of making the argument explicit under uncertainty, and the right response to a violation is to name it and defend it, not to pretend it passes.
implementationWhether an internal DSL is a real option depends on the host language: Lisp, Racket, Ruby, Kotlin, Rust and Scala can carry substantial notation, while Java and Go can carry a builder API and little more. A recommendation to "start internal" is only actionable once the host is named, and in a host with no syntactic flexibility the ladder effectively skips a rung.
specAdopting an existing embeddable language rather than building one changes the calculus and is frequently the best available answer: CEL, Rego, Starlark, Jsonnet and Lua are all specified, sandboxable and come with tooling and a community. The tradeoff is that their semantics are fixed and their restrictions are theirs rather than yours, so a domain guarantee you wanted may not be expressible.

If you were asked this in an interview

  • A team wants to replace a config module with a small language. What do you ask them?
  • What would change your answer from no to yes?
  • Why is a syntax decision harder to reverse than an API decision?

Connections

Domains that do not exist yet
  • DevOps / Production Engineering — Who owns a language once it is load-bearing in the build or the deploy path
    The decisive criterion in this lesson is whether the tooling has a funded owner, and that is an organisational question rather than a technical one. What happens when the owner leaves, how a language becomes unmaintained infrastructure, and how such systems are decommissioned belong to production engineering.