Queriesfilteringquery parametersallowlistindexeslist endpoints

Filtering: An Allowlist With an Index Bill

Every filter parameter is a promise that a class of database queries will stay fast forever. Explicit, typed, allowlisted filters keep that promise affordable; a generic query language hands your query planner to strangers.

Follow the failure

Frame the contract

API design starts with a consumer, a design question and a guarantee — never with a URL.

Design question
Which subsets of this collection does the contract promise to serve efficiently — and which combinations did you just promise by accident?
Consumers
A UI's filter sidebar (status, date range, assignee); a partner's reconciliation job fetching `status=failed&created_after=…`; an analyst's script that will try every parameter combination the docs mention, plus a few they guess.
The promise
A well-designed filter contract enumerates the filterable fields and operators explicitly, validates them like any other input, keeps every supported combination within an index's reach, and rejects what it never promised — instead of serving it with a table scan.
RequirementConsumersResource ModelStyleContractValidationAuthorizationErrorsIdempotencyPaginationVersioningObservabilityEvolutionTrade-offs

A filter parameter is a standing query contract

When the docs say GET /orders?status=paid&created_after=2026-01-01, they are not describing a convenience — they are promising that this *shape of query* is a supported operation at production scale, indefinitely. The client cannot know or care whether the predicate is served by an index seek or a 40-million-row scan; the contract made both look identical. Which means the filter list is really a list of query plans you have agreed to fund, and every field you add to it adds rows to that bill (see What an API Contract Actually Is).

This is why the design position is an allowlist: a small, explicit set of filterable fields, each with defined operators (status equals one of an enum; created_after/created_before bound a range) and each backed by a deliberate index decision. The opposite default — "we pass query params through to the WHERE clause" — ships fast and fails in two directions at once: performance (any unindexed field becomes a scan endpoint the moment someone filters on it) and security (client-controlled field names flowing toward SQL is the posture injection lives in, even when your ORM technically escapes everything).

Filters are input, so they get the full Validation Errors: Feedback, Not Verdicts treatment: typed values (created_after must parse as a date — a silently ignored typo'd date returns the *unfiltered* collection, which for a reconciliation job is a correctness bug, not a UX flaw), enum values checked against the enum, and unknown filter fields rejected, not ignored. Silently ignoring ?staus=paid returns every order while the client believes it asked for a subset — the most expensive kind of wrong answer, an all-2xx one.

Pass-through filtering: the client writes your WHERE clause
1GET /orders?filter[shipping_address.zip]=90210
2GET /orders?filter[internal_margin][gt]=0.4
3GET /orders?staus=paid200, ALL orders (typo ignored)
4
5# any column is filterable → any column is a
6# promised query plan → the client just chose
7# a 40M-row scan; also: internal fields leak,
8# and field names flow toward the query layer
Allowlisted, typed, rejected-when-unknown
1GET /orders?status=paid&created_after=2026-01-01
2
3Filterable (documented, exhaustive):
4 status eq enum: pending|paid|refunded
5 created_after range RFC 3339 timestamp
6 created_before range RFC 3339 timestamp
7 customer_id eq id
8
9GET /orders?staus=paid
10400 { "code": "validation_failed",
11 "details": { "fields": [ { "path": "staus",
12 "rule": "unknown_parameter" } ] } }

The good side is smaller and that is the point: four fields, each typed, each index-backed, each documented. The bad side's flexibility is a blank check — on performance (scans), on security (field names as input), and on evolution (every internal column that ever worked in a filter is now load-bearing).

The combinatorics: promises multiply, indexes do not

The subtle trap is combinations. Four filterable fields plus a sort order look like four promises; they are closer to dozens, because clients combine them: status + created_after, customer_id + status + sort by total… A composite index on (status, created_at) serves the first beautifully and does nothing for customer_id + total. Each *combination pattern* is its own query plan, and B-tree index mechanics (leftmost-prefix rules, one range condition, sort direction) decide which combinations one index can cover — the contract you can afford is downstream of index design, which is why filter design is a conversation with whoever owns the schema (see Cursor Pagination: An Opaque Bookmark, Not a Position for the same bill arriving via ordering).

You do not need an index per combination — you need a *policy* for combinations. The workable options: keep the filterable set small enough that the handful of real combinations are each covered (the usual answer); designate some filters as always-required so the index prefix is guaranteed (customer_id required on a per-tenant API — which is also the authorization boundary doing double duty, see Authorization Design in the Contract); or explicitly tier the contract — "these combinations are fast; anything else may be slow or rejected". What is not workable is silence, where the docs imply every combination is equal and the database disagrees at 2 a.m.

Filters also interact with Pagination: Choosing How Lists End in one way worth engineering deliberately: the filter set is part of the traversal's identity. A cursor minted under status=paid must not continue under status=refunded — bind the filter hash into the cursor and reject mismatches (see Cursor Pagination: An Opaque Bookmark, Not a Position). And every filtered traversal still needs the total ordering and tiebreaker; a filter that shrinks the result does not excuse the ordering contract.

Filter-surface policies, priced
PolicyConsumer getsProvider paysFails when
Small allowlist, all combinations indexedPredictable speed everywhereA few composite indexes; saying "no" oftenConsumers genuinely need ad-hoc queries
Required tenant/anchor filter + allowlistFast within their partitionOne index family; contract stiffnessCross-tenant/admin queries need a separate path
Tiered: fast set + documented slow setFlexibility with honest labelsTwo SLOs; complexity in docs and limitsClients ignore the labels and build on the slow tier
Generic query language (?q=field:op:value…)Arbitrary queriesQuery-cost control, planner babysitting, injection surfaceAny consumer discovers an expensive shape and automates it

When consumers really do need arbitrary queries

Sometimes the requirement is genuinely open-ended — an analytics surface, an admin power-search, an integration platform. The answer is still not "pass params to the WHERE clause"; it is recognizing that you are now building a *query API*, a different product with different machinery: a defined grammar with a parser (never string concatenation), per-field capability declarations, query cost estimation with rejection of expensive shapes, tight result and time limits, and usually a separate datastore fed by replication so the analyst's five-way filter never competes with checkout for the primary's buffer pool. What GraphQL Costs tells the same story from the GraphQL side; Search Is a Different Contract Than Filtering tells it for relevance-ranked text.

The decision discipline: start from measured consumer tasks, not imagined flexibility. Filter parameters are trivially additive — shipping four fields today and adding a fifth next quarter when telemetry shows clients client-side-filtering on it is cheap (see the observe list below). Shipping twenty speculative fields today means twenty standing promises, most unused, some unindexed, all load-bearing the moment one consumer automates against them. Filters are the clearest case in API design where the small contract is the strong one.

  • Open-ended querying is a product, not a parameter: grammar, cost control, limits, often a replica-fed store.
  • Filters are additive — ship the measured few, add on demand; removal is the breaking direction (see Backward Compatibility: The Real Rules).
  • Watch what clients fetch-then-discard: client-side filtering is the demand signal for the next filter field.
  • Filterable ≠ returnable: a field can be in responses but not filterable — and occasionally filterable but redacted in responses (existence is then still leaked; decide deliberately).

Key points

  • Every filter parameter is a standing promise that a query shape stays fast at scale; the filter list is a list of query plans you agreed to fund.
  • Allowlist fields and operators explicitly; validate types and enums; reject unknown parameters — a silently ignored typo returns the unfiltered collection as an all-2xx wrong answer.
  • Combinations are the real contract surface: composite-index mechanics decide which combinations one index covers, so filter design is co-design with the schema owner.
  • Have a stated combination policy — small-and-covered, required anchor filter, or honest tiers — instead of implied uniformity the database cannot deliver.
  • Bind filters into cursor identity so traversals cannot continue under changed predicates.
  • Genuinely open-ended querying is a separate product with a parser, cost control and usually a separate datastore — not a pass-through WHERE clause.

Follow the failure

How the contract fails or gets misused, hop by hop — and what it costs when it completes.

  1. 1
    Team → API: maps query params straight onto the ORM's where() — every column filterable, zero docs needed, ships in a day.
  2. 2
    Consumers → API: discover filtering works on anything; dashboards and scripts accrete filters on unindexed columns.
  3. 3
    Collection → scale: a partner automates ?refund_reason=… (no index) hourly; each call is a full scan competing with production writes.
  4. 4
    Provider → database: p99 degrades API-wide; the slow-query log is a catalog of filters nobody designed.
  5. 5
    Team → cleanup: tries to restrict the filter set and discovers every accidental filter has a consumer — the allowlist now requires a deprecation program.
What breaks
  • One consumer's creative filter degrades every consumer's latency — the blast radius of a table scan is the whole database, not the one endpoint.
  • Silently ignored parameters corrupt downstream data: clients process unfiltered collections believing they asked for subsets.
  • Internal schema leaks through filterable columns: renaming a column becomes a breaking change to filters nobody knew existed.

Design, observe, evolve

A contract decision is incomplete until you know how you would notice it failing and how it changes later.

Design the contract
  • • Enumerate filterable fields and operators in the contract, generate validation from that enumeration, and reject unknown parameters with a field-level error.
  • • Back every documented filter combination with an index decision before it ships; make "which index serves this?" a required line in the endpoint review.
  • • Anchor per-tenant APIs with a required filter that doubles as the authorization boundary and guarantees the index prefix.
  • • Cap result windows and execution time on filtered queries regardless — limits are the backstop for the combination you missed (see [[unbounded-collections]]).
Observe in production
  • • Log filter-shape usage (which fields, which combinations) per consumer: it is simultaneously your index worklist, your deprecation evidence, and your next-filter demand signal.
  • • Alert on slow-query-log entries originating from filter endpoints — each one is a contract promise the database is failing to keep.
  • • Track `unknown_parameter` rejections: a cluster on one name (`staus`, `state`) is a docs or SDK bug; a scatter of internal column names is someone probing your schema.
Evolve without breaking
  • • Adding filter fields and operators is additive and safe; do it on measured demand rather than speculation.
  • • Removing or restricting a filter is a breaking change requiring usage telemetry and a deprecation window — accidental filters have consumers too (see [[consumer-driven-evolution]]).
  • • When ad-hoc demand outgrows the allowlist, add the query product (tiered or separate endpoint, replica-backed) beside the fast path instead of loosening the fast path's promises.
What it costs
  • • Allowlists mean telling consumers "no" — repeatedly, with each "no" a small product negotiation the pass-through design never has (until it has a database incident instead).
  • • Every supported combination costs a composite index: storage, write amplification, migration time; the filter surface is bounded by the write budget, not imagination.
  • • A required anchor filter simplifies indexes and authorization but stiffens the contract — the legitimate cross-partition consumer needs a second, deliberately-designed path.

Misconceptions

Claim
“More filterable fields is strictly better API design — flexibility is free.”
Reality
Each field is a standing query-plan promise with an index bill, a validation surface and a removal cost. Unused flexibility is not free; it is unfunded liability that a single automated consumer converts into a scan workload.
Claim
“The ORM escapes everything, so pass-through filters are safe.”
Reality
Escaping stops classic injection of *values*; it does nothing about client-controlled *field names* selecting scan targets, filtering on privileged columns (?is_admin=true as an oracle), or the DoS of unindexed predicates. The allowlist is the control; escaping is table stakes.
Claim
“Ignoring unknown parameters is friendlier than rejecting them.”
Reality
For filters, silence inverts the meaning of the response: the client asked for status=failed (typo'd) and got *everything*, 200 OK. A rejected typo costs one round trip; an ignored one can cost a partner a corrupted reconciliation run. Reject unknown filter parameters specifically, whatever your body-field policy is.