Plugin Architecture
A plugin system is right when independent parties genuinely must extend you. What it costs — a frozen API, a lifecycle, isolation and compatibility — is permanent.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.
We want third parties to extend the product. What are we actually signing up for?
Sales has committed that customers can write their own validation rules and ship them into the platform without waiting for our release cycle. Two enterprise deals depend on it.
Define a Plugin interface, load classes from a directory at startup, and call them at the right moment. It is the strategy pattern with dynamic loading — a week of work.
The week builds the loading. The permanent costs are everything after: the API can never change incompatibly again, and you now own the compatibility of code you did not write and cannot read (API Stability).
- The week builds the loading. The permanent costs are everything after: the API can never change incompatibly again, and you now own the compatibility of code you did not write and cannot read (API Stability).
- In-process plugins share your heap, your threads and your credentials. An infinite loop in a plugin is a platform outage; a plugin reading a global connection pool is a tenancy breach (Capability Passing).
- Failure attribution collapses. Every incident starts with "is it us or a plugin", and if the answer is not obvious from telemetry, mean time to diagnosis doubles for every incident, including the ones plugins had nothing to do with.
- Plugin authors will depend on whatever they can reach, not on what you documented, so the effective API is the transitive closure of everything reachable from the objects you hand them (Encapsulation Radius).
What limits the solution, and what must never stop being true
This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.
- Plugins run inside our process on our infrastructure, so a plugin fault is our outage and a plugin's CPU is our bill (Least Privilege as a Design Decision).
- We deploy weekly; customers upgrade when they feel like it, so several plugin API versions are live at once (Versioned Interfaces).
- We cannot see plugin source code, and we will be the ones paged when one misbehaves (Debuggability by Design).
- A plugin can never observe or modify another tenant's data, whatever it does — this is a trust boundary before it is a module boundary (Trust Boundaries).
- A misbehaving plugin degrades one customer, never the platform.
- Core invariants stay enforced by core code. A plugin can reject, it can annotate, and it cannot be the thing that guarantees a rule holds (Enforcing Invariants).
Who owns what, and where the seams fall
Responsibilities decide boundaries; boundaries decide what an interface has to say.
- The host owns the lifecycle, the isolation, the resource limits and the enforcement of every core invariant.
- The host owns a versioned, deliberately narrow API surface, and owns deprecating it on a published schedule (Deprecation).
- The plugin owns its own logic and owns nothing about ordering, persistence, or what happens when it fails.
- Somebody in the organisation owns plugin compatibility as ongoing work, forever. If nobody is named, it is the on-call engineer by default.
- The boundary is a trust boundary before it is a design boundary: everything crossing it is untrusted input, including return values (Trust Boundaries).
- Pass capabilities, not context. A plugin should receive exactly the data and exactly the operations it needs, never a service locator or a whole request context (Capability Passing).
- The API should be data-in, data-out wherever possible. Handing over a live domain object hands over everything reachable through it.
Four isolation levels, and what each actually costs
The first decision is not what the API looks like — it is where the plugin runs, because that determines what a bad plugin can do to you and how much of the safety work you have to build yourself. Teams tend to choose in-process by default because it is the easiest to prototype, which is the one property that stops mattering after week one.
Note that the option with the best isolation is also the one least often called a plugin architecture. A webhook to code you never host gives you process, machine and blast-radius isolation for free, and it costs a network round trip.
| Option | Simplicity | Flexibility | Performance | Testability | Operational | Migration cost | Note |
|---|---|---|---|---|---|---|---|
| In-process, same runtime | Fastest to build and fastest at runtime. A plugin can exhaust your heap, block your threads and reach anything your process can reach; every safety property must be built by you, and API changes are the hardest to roll out because everything shares one deployment. | ||||||
| In-process sandbox (WASM or a restricted interpreter) | Real memory and capability isolation with no network hop, and resource limits the runtime enforces rather than you. Costs a restricted programming model that some legitimate use cases will not fit, plus a marshalling layer for everything crossing the boundary. | ||||||
| Separate process, supervised | Faults, CPU and memory are contained by the operating system, which is decades more reliable than anything you will write. Costs serialisation on every call, a supervision and restart story, and deployment of code you do not control. | ||||||
| Webhook — the customer hosts it | Isolation is total and free: their outage, their bill, their runtime, their language. Costs network latency and an at-least-once delivery model, and you cannot support synchronous extension points that must return a decision inside a request (Partial Failure). |
caveat The scores compare mechanisms and say nothing about whether you should have plugins at all, which is the larger question and is answered by the customer contracts, not by this table. They also assume you are equally competent at all four, which is unlikely: a team that already runs a supervised worker fleet finds the third row much cheaper than these numbers suggest, and a team that has never sandboxed anything finds the second row far more expensive. Operational cost here means steady-state, not the build.
The lifecycle you have to build anyway
The part teams underestimate is not calling the plugin — it is everything around the call. A plugin has to be discovered, validated against an API version, loaded, made active, observed, degraded when it misbehaves, disabled without a deploy, and upgraded while requests are in flight.
Writing this out as a state machine early is worth an afternoon, because the transitions you leave reachable become the incidents you have at 3am. In particular, "active" must be leavable without a deployment, and "failing" must not be a state a plugin can stay in indefinitely while quietly returning nothing.
| From | On | To | Guard | Effect |
|---|---|---|---|---|
| Discovered | validate | Rejected | unsupported API version, bad signature, or excess capability request | Log with the plugin id and the specific reason; notify the customer, never silently ignore. |
| Discovered | validate | Loaded | manifest valid and API version supported | Allocate the isolation unit and grant exactly the declared capabilities. |
| Loaded | enable | Active | health probe passed within the startup budget | Begin routing calls; emit a version-tagged activation event. |
| Active | error rate or timeout threshold exceeded | Degraded | — | Open the circuit breaker and fall back to the host default for the extension point. |
| Degraded | probe succeeds after backoff | Active | a bounded number of consecutive successes | Close the breaker gradually rather than all at once (Retries Are a Property of the Operation). |
| Degraded | repeated failure to recover | Disabled | — | Stop calling entirely and raise a ticket against the plugin owner. |
| Disabled | operator re-enables | Loaded | a human decision, not an automatic timer | — |
| Active | operator kill switch | Disabled | — | Must work without a deployment. This is the transition you will need during an incident. |
- Discovered → Active — Skipping load and health probe means the first evidence that a plugin is broken is a customer request failing. Validation exists to move that discovery before traffic.
- Rejected → Loaded — A plugin that failed signature or capability validation must not be loadable by a retry, or the check is advisory. Rejection is terminal for that artefact; a fixed plugin is a new artefact.
- Degraded → Degraded — A self-loop with no bound is how a plugin sits half-broken for weeks. Degradation must be time-boxed and must terminate in Active or Disabled.
- Disabled → Active — Re-enabling must go back through load and health probe. Jumping straight to Active reuses state from before the failure, which is exactly the state that failed.
The two transitions that get skipped in the first implementation are the operator kill switch and the bound on Degraded, and they are the two you need during the first incident. Building them later means building them under pressure.
What goes wrong, and who gets paged
Every failure here has the same shape: something you cannot read does something you did not anticipate, and your telemetry has to make the attribution obvious. If a dashboard cannot answer "which plugin" in one glance, every incident costs an extra half hour before anyone starts fixing anything.
Note the last row especially. The most expensive plugin failures are not crashes — they are plugins that succeed while quietly meaning something different, because those are invisible until a customer notices in their own data.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Plugin never returns | Request threads drain; unrelated endpoints slow down | A synchronous in-process call with no deadline | A per-call timeout enforced by the host, not requested of the plugin, plus a bulkhead so plugin calls cannot consume the whole pool (Bulkheads). |
| Plugin allocates without bound | Host memory climbs over hours and the process is killed | Shared heap, no ceiling | Resource limits at the isolation boundary. In-process, this is largely unachievable — which is an argument about isolation level, not about limits (Resource Limits). |
| Plugin mutates a domain object it was handed | A core invariant is violated in code with no plugin in the stack trace | A live aggregate crossed the boundary instead of a copy | Pass immutable data across the boundary and re-validate everything returned (Immutability). |
| Plugin reaches a global the API never mentioned | Cross-tenant data appears in a plugin's output | Ambient access to a connection pool or a request context | Capability passing: hand over exactly the operations granted, and make the ambient path unreachable (Capability Passing). |
| Host ships a compatible-looking change | Plugins break on a patch release | Authors depended on behaviour that was never part of the contract | A compatibility suite run against supported versions, plus a canary population of real plugins before a wide rollout (Canary Deployments). |
| Plugin returns plausible but wrong results | No error anywhere; a customer reports bad data weeks later | The host trusts the return value as thoroughly as it distrusts the call | Validate returns against the schema and the domain rules, and keep the core invariant enforced in core code — the plugin advises, it does not decide (Enforcing Invariants). |
How to build it
Most important first.
- Decide the isolation level first, because it determines everything else: same process, separate process, WASM sandbox, or a webhook to code you never host at all. The last option is a plugin architecture and is usually the cheapest one that works (What Changes at the Network Boundary).
- Define the extension points narrowly, in terms of the domain rather than your internals: "validate this submission and return findings", not "here is the request and the ORM session".
- Version the API from day one, even at version one, and define what a plugin declaring an old version gets (Semantic Versioning).
- Give plugins an explicit lifecycle with states you can observe — discovered, loaded, active, degraded, disabled — and make disabling a first-class operation rather than an incident response (State Machines).
- Budget the resources: a timeout, a memory ceiling, a call-rate limit, and a circuit breaker that disables a plugin that keeps failing (Designing for Failure).
- Ship the compatibility test kit with the API, so plugin authors can find out they are broken before your customers do (Contract Tests).
What the next change costs
The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.
- Every future change to core internals is now filtered through "does this move something a plugin can reach". That filter applies to changes with nothing to do with plugins, which is where most of the ongoing cost lands.
- An incompatible API change costs a deprecation window measured in quarters, a migration guide, outreach to authors you may not know, and a period of supporting both versions (Expand and Contract).
- Adding a new extension point is cheap; removing one is close to impossible, so each one is effectively permanent from the day it ships.
- Every incident acquires an extra diagnostic step forever. That is a small per-incident cost multiplied by every incident for the life of the product, and it is routinely left out of the estimate.
- A plugin API is a product with a permanent maintenance budget, not a design pattern. The honest comparison is against hiring someone to build the integrations directly, which is often cheaper for the first dozen.
- Isolation costs latency and operational complexity — separate processes mean serialisation, supervision and deployment; a sandbox means a restricted runtime that will not do what someone needs.
- Every safety measure you add reduces what plugins can do, and the customers who wanted plugins wanted them precisely to do unusual things.
What can go wrong
- A plugin blocks, and the host thread pool drains. Everything is now slow, and the dashboards blame the host.
- A plugin holds a reference to something long-lived and leaks memory over days, so the failure surfaces in a host process restart far from its cause (Hidden Global State).
- A plugin mutates a domain object it was handed for reading, and a core invariant is violated by code you cannot see (Immutability).
- The API is changed "compatibly" and a plugin depending on undocumented behaviour breaks, which is your problem regardless of what the contract said.
- The mitigation fails on its own terms: a sandbox is added, it is too restrictive for a paying customer's real use case, and an escape hatch is granted to one plugin — after which the sandbox describes an intention rather than a boundary.
- The host depends on nothing in any plugin, which is the property the whole design exists to preserve — the moment core logic requires a plugin to be present, the architecture has inverted (Dependency Direction).
- Plugins depend on the API version they declared, and transitively on any type reachable through it.
- You acquire a dependency on your own past: every version you have ever shipped is a thing you support until you deprecate it publicly.
- "We should build a plugin system so the code is modular." Modularity is achieved with internal module boundaries at a fraction of the cost. A plugin API is for parties who cannot change your code, and if there are none, you have paid for a boundary and got a strategy pattern (Internal Module Contracts).
- "Plugins make us extensible." They make *other people* able to extend you. Your own team was always able to; you have bought a distribution property, not a design property (Extensibility).
- "We can lock it down later." Every restriction added after plugins exist breaks someone who is paying you, so the restriction is negotiated rather than decided (Least Privilege as a Design Decision).
- "Webhooks are not a plugin architecture." They are one, with a process boundary and a network in the middle, and they solve the isolation problem by construction. Dismissing them on aesthetic grounds is how teams end up hosting untrusted code for no reason (What Changes at the Network Boundary).
Testing it, and how it ages
- A compatibility suite the host runs against every supported API version, so "we did not break v1" is asserted rather than believed (Contract Tests).
- Adversarial plugin tests in CI: one that never returns, one that throws, one that allocates without limit, one that tries to reach a neighbouring tenant. Each must degrade exactly one customer (Failure-Aware Feature Design).
- Test the lifecycle transitions themselves, especially disable-while-running, which is the path exercised during an incident and the least exercised in development (State Machines).
- The API surface only grows. Every quarter someone needs one more thing, and each addition is individually reasonable and permanent (Do We Need a Package for This?).
- Isolation tends to tighten over time and only after an incident, which is the expensive order to do it in — the cheap moment is before any plugin exists.
- Successful plugin ecosystems eventually force a v2 with a migration path; the ones that never manage it accumulate compatibility shims until the host is difficult to change at all.
Where this applies
This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.
- GENERALThat an extension point usable by parties who cannot change your code becomes a permanent contract, with versioning, isolation and compatibility obligations, holds for any host language or runtime — only the isolation mechanism differs.
- DOMAIN-SPECIFICFor developer tools, editors, CI systems and marketplaces the plugin API is the product, and the costs listed here are simply the cost of doing business. For a line-of-business SaaS with fifty customers, the same architecture is usually a very expensive way to avoid writing fifty integrations, and the honest alternative is a webhook plus a well-documented API.
- CONTESTEDThe strongest argument for building it early: an ecosystem is a compounding advantage, and a plugin API retrofitted onto a mature codebase is enormously harder than one designed in from the start, because by then nothing is shaped to be safely exposed. Teams who have built successful platforms genuinely believe this, and they are right about the retrofit difficulty. The counter is survivorship — the platforms that succeeded are visible, the far larger number that built an API nobody wrote against are not, and their cost was paid every sprint regardless.
Where the depth lives
This domain teaches the codebase-level structure and hands the rest off.
- — System Design — running untrusted extensions is a capacity and blast-radius problem as much as a code-structure one, and the isolation choice here is the same choice as scheduling untrusted workloads on shared infrastructure.
- — Testing & Reliability Engineering — a compatibility suite against every supported API version is the mechanism that makes the promise real, and its coverage strategy is a testing problem this lesson assumes rather than solves.