RAGfilteringtenant isolationaccess controlrecencypre-filtering

Metadata Filtering

Restricting retrieval by tenant, permission, recency, or type is done with metadata filters — and where the filter runs decides both correctness and recall.

Interview question
Progress

Filters are part of retrieval, not a decoration

A retriever that returns the 5 most similar chunks in the whole corpus is rarely what you want. The real query is "the 5 most similar chunks *among those this user is allowed to see, in this workspace, from the current version of the docs*". Those constraints are expressed as predicates over chunk metadata: tenant_id = "acme", acl && user.groups, updated_at > now() - 90d, doc_type IN ("policy", "runbook").

This is why Ingestion: Parsing & Chunking insists on rich metadata: every filter you will ever need must be a field on the chunk. Filtering by something you did not store means re-ingesting.

Pre-filtering vs post-filtering

Post-filtering runs the ANN search first (top-k across everything), then drops results that fail the predicate. It is simple and fast, and it is wrong whenever the filter is selective: if tenant acme owns 1% of chunks and you fetch top-20, on average 0.2 of them belong to acme — the user gets zero or one result even though acme has hundreds of relevant chunks. Over-fetching (top-2000 then filter) papers over this at the cost of latency and still fails for very small tenants.

Pre-filtering applies the predicate first and searches nearest neighbours only within the surviving set. This is exact with respect to the filter but interacts badly with graph indexes: HNSW navigates neighbour links, and if most neighbours are filtered out the greedy walk gets stuck and recall collapses. Stores handle this differently — some fall back to brute force when the filter is selective, some build per-partition indexes, some integrate the filter into the graph traversal. In pgvector, a partial index or a partitioned table per tenant sidesteps the problem entirely.

The practical rule: know your filter selectivity. For low-selectivity filters (exclude 10% of chunks) post-filtering with modest over-fetch is fine. For high-selectivity filters (a tenant, a single document) use pre-filtering, partitioning, or a separate index per partition. Always test recall with the real filter, because unfiltered benchmark numbers tell you nothing.

Pre- vs post-filtering
yesnoQuery + filterFilter selective?Apply filter firstANN over-fetch top-NANN within subsetDrop non-matchingTop-k results
UserLLMAgentToolDataDecisionHumanGuardrail

Tenant isolation

In a multi-tenant product, a retrieval that returns another tenant's chunk is a data breach, full stop. Tenant isolation must be enforced *in the retrieval query* — never by asking the LLM to ignore passages it should not have seen. The filter must be applied on every retriever in a hybrid setup, on every index, and it must be impossible to omit: derive tenant_id from the authenticated session, not from the request body, and make the retriever function require it as a non-optional argument.

Stronger isolation options, in increasing cost: a mandatory WHERE tenant_id = $1 plus row-level security in the database; a separate index or partition per tenant; a separate database per tenant for regulated customers. Row-level security is attractive because it enforces the filter even when an engineer forgets it in a new code path.

Access control at retrieval time

Within a tenant, documents have permissions: HR docs for HR, a customer's contract for its account team. Model this as an acl array on each chunk (group ids or principal ids, mirrored from the source system) and filter with acl overlaps user.groups. Sync the ACLs when the source changes, because a document made private yesterday must stop being retrievable today — and remember that the index is a *copy* of the data with its own permissions to maintain.

Two subtleties. First, the LLM's answer can leak content even when a chunk is technically permitted, if the chunk itself quotes something sensitive — ACLs on chunks are only as good as ACLs on the source. Second, never fall back to "no filter" when the user's groups cannot be resolved; fail closed and return nothing.

A retriever that cannot be called without a tenant and a principal.
1def retrieve(query: str, *, tenant_id: str, groups: list[str], k: int = 20,
2             since: datetime | None = None) -> list[Chunk]:
3    if not tenant_id or not groups:
4        return []                       # fail closed, never "no filter"
5    where = ["tenant_id = %(t)s", "acl && %(g)s"]
6    params = {"t": tenant_id, "g": groups, "v": embed(query), "k": k}
7    if since is not None:
8        where.append("updated_at >= %(since)s")
9        params["since"] = since
10    sql = f"""
11        SELECT id, doc_id, text, updated_at, 1 - (embedding <=> %(v)s) AS score
12        FROM chunks WHERE {' AND '.join(where)}
13        ORDER BY embedding <=> %(v)s LIMIT %(k)s"""
14    return db.query(sql, params)

Recency and other soft filters

Not every constraint is a hard predicate. "Prefer recent docs" should usually be a boost, not a cutoff: multiply the similarity score by a decay such as 0.5 ** (age_days / 180), or add a recency term during Reranking. A hard updated_at > 1 year cutoff hides the one runbook nobody has touched because it still works.

Hard filters are for correctness (tenant, ACL, document version, language); soft boosts are for relevance (recency, popularity, doc type preference). Keep the distinction explicit in code so a relevance tweak can never weaken a security guarantee.

Key points

  • Filters are predicates over chunk metadata; store every field you will ever filter on.
  • Post-filtering fails on selective filters; pre-filtering fights graph indexes — measure recall with the real filter.
  • Tenant isolation is enforced in the query and derived from the session, never left to the model.
  • ACLs are mirrored onto chunks and synced; fail closed when permissions cannot be resolved.
  • Apply the same filters to every retriever in a hybrid setup.
  • Hard filters for correctness and security; soft boosts for recency and relevance.

When to use — and when not to

Use it when
  • Any multi-tenant deployment.
  • Corpora with per-document permissions.
  • Versioned docs where only the current version should answer.
  • Scoping queries to a doc type, product, or language.
Avoid it when
  • Recency as a hard cutoff — use a boost unless staleness is a correctness issue.
  • Filtering by something not stored on the chunk — re-ingest first.
  • Relying on prompt instructions to hide content the retriever already returned.

Failure modes

  • Post-filtering returns empty results for small tenants.
  • HNSW recall collapses under a selective pre-filter and nobody measured it.
  • Filter present on the dense retriever, missing on BM25; cross-tenant leak.
  • ACL changes in the source system never reach the index.
  • Permission lookup fails and the code falls back to unfiltered retrieval.
← Dense, Sparse & Hybrid RetrievalReranking →