Securing Hybrid Search in RAG: Why You Must Filter Unsafe Data Before Reciprocal Rank Fusion

15min read • 2026-11-19Data IconData
Illustration conceptuelle de la sécurisation de la recherche hybride dans une architecture RAG

Reciprocal Rank Fusion

What this article covers: A step-by-step walkthrough of how we designed and implemented Role-Based Access Control into every layer of a production Retrieval-Augmented Generation system — from document ingestion to chunk retrieval to the HTTP response. Every architectural decision is explained with the reasoning behind it, the alternatives we considered, and the failure modes we were guarding against.

Key Takeaways

  • Defense in Depth: RAG has 5+ access control fail points—you must secure every single layer individually.
  • Filter Before Ranking: Apply permission filters before ranking to protect against data leakage and preserve accurate relevance.
  • Public Access by Default: Treat "public" as a baseline permission for all logged-in users, not a separate edge case.
  • Fail-Safe Defaults: Unrecognized roles must automatically fall back to the lowest access level (public).
  • Scope Note: This covers retrieval security only—for prompt injections and jailbreaks, check out Building Defense-in-Depth for Enterprise RAG. 

1. Why RBAC in RAG Is a Fundamentally Different Problem

Most engineers encounter RBAC first in the context of databases or APIs: a user makes a request, the server checks their role against a permission table, and either serves the data or returns 403. The access control boundary is clean and well-defined.

RAG breaks this model entirely.

In a RAG pipeline, a document does not stay a document. It gets shredded into hundreds of chunks, each chunk gets transformed into a high-dimensional embedding vector, and those vectors get stored in a specialized index alongside metadata. When a user asks a question, our system by design runs two parallel searches — one dense semantic search over the vector index, one sparse keyword search over a BM25 index — fuses the results with a ranking algorithm, and then feeds the top chunks as context to a language model before generating an answer.

There is no single access boundary. There are at least five places where a privileged chunk can leak:

  • At ingestion: if a document is stored without a role tag, all downstream chunks are unclassified and effectively public.
  • At the dense retrieval layer: if the vector store query does not include a role filter, semantically similar but unauthorized chunks are returned.
  • At the sparse retrieval layer: if BM25 results are not post-filtered, keyword-matching unauthorized chunks pass through.
  • At fusion: if role filtering happens after Reciprocal Rank Fusion, an unauthorized chunk can influence the ranking of authorized ones before being discarded.
  • At the LLM context: if any of the above fail, the model sees content it should not, and the answer may reflect it even if we try to strip the source citation.

The architecture we built treats each of these as a hard enforcement point, not a best-effort filter: RBAC is attached during ingestion, resolved before retrieval, enforced independently in both search paths, and preserved all the way into the prompt. The five enforcement points above all concern access control at the retrieval layer — who can see which chunks, and at which stage of the pipeline. They do not cover attacks that target the LLM itself after authorized content has been passed to it. Direct and indirect prompt injection, jailbreaks, and model output manipulation are a separate threat class and are addressed in our companion article:  Building Defense-in-Depth for Enterprise RAG. The two articles are designed to be read together as a complete security architecture for production RAG systems.

2. Designing the Role Set: Explicit, Ordered, and Defensively Parsed

Before writing retrieval logic, we defined a rigid role set. These roles become deeply embedded in the database schema, vector filters, and audit logs; changing them later is incredibly expensive. 

Priority Ordering for Multi-Role Resolution

When a user holds multiple roles and we need to report a single "effective" role — for audit logs, for display in the admin UI, for a default label — we use a deterministic priority ordering, for example:

admin > engineering > finance > hr > support > user > public

The _highest() function iterates this list and returns the first match. This is not used for access decisions (where the full role set matters), only for single-value resolution contexts.

Defensive Defaults Are Not Optional

Two decisions here are worth dwelling on:

DEFAULT_ROLE = user: every new user is seeded with the user role on first login. This means they can access internal-but-not-department-specific content immediately. We chose user over public because most of our employees should have basic internal access on day one, but it is still conservative.

FALLBACK_ROLE = public: whenever the system encounters a role string it does not recognize — from a malformed JWT claim, a database corruption, a deployment misconfiguration — it treats the value as public. It never assumes privilege. This is the security default that matters most: the system fails toward least privilege, not toward open access.

3. The Database Layer: Schema, Pooling, and Atomic Operations

We modeled user-to-role assignments as a simple relational join table. This is deliberately boring, and that is the point. The role store is called on every authenticated request — it must be fast, correct, and predictable.

Schema Design

The core table is a standard many-to-many join between users and roles:

CREATE TABLE user_roles (
    user_id     TEXT        NOT NULL,
    role        TEXT        NOT NULL,
    assigned_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    assigned_by TEXT        NOT NULL,
    CONSTRAINT user_roles_pkey PRIMARY KEY (user_id, role),
    CONSTRAINT user_roles_role_check CHECK (
        role IN ('admin', 'engineering', 'finance', 'hr', 'support', 'user', 'public')
    )
);

CREATE INDEX idx_user_roles_user_id ON user_roles (user_id);

Two design decisions here are worth explaining:

  • The primary key is the pair (user_id, role). This makes duplicate assignments a no-op at the database level — there is no application-side deduplication logic to get wrong.
  • The role column uses a CHECK constraint against the known role set. Any attempt to insert an unrecognized role fails at the database, not silently downstream. This is the first enforcement point of the FALLBACK_ROLE principle described in Section 2 “Designing the Role Set: Explicit, Ordered, and Defensively Parsed”.
  • The trade-off is that adding a department (say, legal) requires a DDL migration. A roles reference table with a foreign key would remove that migration, but on its own it buys less than it seems: the role set also lives in the priority ordering, in visible_to() and in the vector metadata filters, so a new role still ships with a code change. We kept the CHECK because our role set changes rarely and on purpose (see Section 2). For systems where roles are created at runtime, a reference table that the application also loads its role set and priority from is the better fit. 

The “init_user” Pattern:

init_user(email, default_role) is called on every authenticated request for users we have not seen before. It uses a single atomic upsert to avoid race conditions:

Two design decisions here are worth explaining:	
	The primary key is the pair (user_id, role). This makes duplicate assignments a no-op at the database level — there is no application-side deduplication logic to get wrong.
	The role column uses a CHECK constraint against the known role set. Any attempt to insert an unrecognized role fails at the database, not silently downstream. This is the first enforcement point of the FALLBACK_ROLE principle described in Section 2 “Designing the Role Set: Explicit, Ordered, and Defensively Parsed”.
	The trade-off is that adding a department (say, legal) requires a DDL migration. A roles reference table with a foreign key would remove that migration, but on its own it buys less than it seems: the role set also lives in the priority ordering, in visible_to() and in the vector metadata filters, so a new role still ships with a code change. We kept the CHECK because our role set changes rarely and on purpose (see Section 2). For systems where roles are created at runtime, a reference table that the application also loads its role set and priority from is the better fit. 
	The “init_user” Pattern:
	init_user(email, default_role) is called on every authenticated request for users we have not seen before. It uses a single atomic upsert 		to avoid race conditions:

The ON CONFLICT DO NOTHING clause handles the case where two concurrent requests arrive for the same new user simultaneously. The second insert silently succeeds without duplicating the row. Both requests then read the same consistent role set from get_user_roles. There is never a window where a new user is ungoverned, and there is no risk of duplicate role assignments from concurrent logins.

A note on JWT lifetime

Role assignments are resolved against the database on every request, not cached in the token. This is correct behavior — but it assumes tokens are short-lived. A JWT valid for 24 hours issued to a user whose role was subsequently revoked will carry stale claims until it expires.

Recommendation: Keep token lifetimes at 15 minutes or less, or implement a token revocation mechanism (a Redis blocklist keyed by jti claim is a common pattern). Long-lived tokens without revocation create a window where a role downgrade is not enforced. 

Managing role assignments and revocation

Role assignments should change only through a small admin API, never through ad-hoc database writes. Three operations are enough: list a user’s roles (GET /admin/users/{user_id}/roles), grant a role (POST to the same path), and revoke one (DELETE /admin/users/{user_id}/roles/{role}). Any admin UI sits on top of these endpoints and adds no access logic of its own.

The endpoints are protected by the same role model they manage: only callers holding admin can use them, and that check goes through the same role lookup as every other request. Each grant writes the caller’s identity to assigned_by, so the table itself answers who granted what and when. Unknown roles are rejected by the CHECK constraint, so a typo in the admin UI fails loudly instead of creating a role that no filter ever matches. Every grant and revoke should also be written to the audit log as its own event, because a revoke deletes the row and leaves nothing behind to inspect later.

Revocation is where token lifetime matters. Because roles are resolved from user_roles on every request, a revoke takes effect on the user’s next request for every decision the database makes. The remaining exposure is anything read from the token itself. On revoke, the API should therefore also add the jti of the user’s active tokens to the Redis blocklist (which means recording issued jti values per user), with a TTL equal to each token’s remaining lifetime so the blocklist cleans itself up once those tokens would have expired anyway. With 15-minute tokens the window is short; with longer-lived tokens the blocklist is the only thing closing it.

A note on lookup cost

Resolving roles on every request costs one indexed lookup returning a handful of rows, which is well under a millisecond on Postgres. At scale the pressure is on connections, not on the query: run it through the application’s connection pool (or a pooler such as PgBouncer) rather than opening a connection per request. If that is still not enough, caching is possible, but each option weakens the revocation guarantee above. A read replica adds replication lag and breaks init_user’s read-after-write, so role lookups should stay on the primary. A short-lived cache (a few seconds, in Redis or in process) turns “a revoke applies on the next request” into “a revoke applies within the TTL”, unless the revoke endpoint also deletes the cached entry. Delete on revoke and keep the TTL as the backstop for a missed invalidation.

4. The Access Logic: Set Theory as Security Policy

The entire runtime access control system reduces to two functions. Understanding them precisely is critical because they are called hundreds of times per request across both retrieval paths.

visible_to: The Core Predicate

visible_to(chunk, user) is the single function that answers the question: can this user see this chunk? It is called for every BM25 result before fusion, and in audit logging.

The logic:

visible_to(chunk C, user U):
  Step 1 → if admin ∈ roles(U)          : return True   (admin bypass)
  Step 2 → if roles(C) ∩ (roles(U) ∪ {public}) ≠ ∅ : return True
  Step 3 →                                 return False

In Python:

ADMIN_ROLE = "admin"
PUBLIC_ROLE = "public"

def visible_to(chunk_roles: set[str], user_roles: set[str]) -> bool:
    """
    Returns True if the user is authorized to see this chunk.

    - Admins bypass all role checks.
    - All authenticated users implicitly hold the 'public' role.
    - Access is granted if the chunk's role set intersects
      with the user's effective role set.
    """
    if ADMIN_ROLE in user_roles:
        return True
    effective_roles = user_roles | {PUBLIC_ROLE}
    return bool(chunk_roles & effective_roles)

Step 1 means admins bypass the intersection check entirely — no role filter is applied at the vector store level either (see allowed_role_filter below).

Step 2 is the key design decision: we always add public to every user's effective role set. Every authenticated user automatically sees public-tagged content without any explicit assignment. There is no special-case branch, no if not roles: return all_public_chunks. The same intersection logic works for every user.

allowed_role_filter: Pre-Computing the Vector Store Filter

def allowed_role_filter(user_roles: set[str]) -> list[str] | None:
    """
    Computes the metadata filter to pass to the vector store before querying.

    Returns:
      - None for admin users: no filter = full index access.
      - A sorted list of allowed roles for all other users.
    """
    if ADMIN_ROLE in user_roles:
        return None
    return sorted(user_roles | {PUBLIC_ROLE})

This function returns:

  • None for admin users — no filter means the vector store returns all chunks regardless of their role tag.
  • sorted(user_roles ∪ {public}) for all others — a sorted list of roles the user is allowed to see, passed as an IN filter to the vector store query.

Why sorted? Consistent ordering makes filter values predictable in logs and simplifies debugging. The sort has no material effect on query planning in the vector stores we tested (Qdrant, Weaviate, Pinecone).

This function is called once per request, before both the dense and sparse retrieval paths. The result is computed once and reused — never recomputed mid-retrieval.

5. Hybrid Retrieval With Dual RBAC Enforcement

Lexical (BM25) and semantic (dense) retrieval fail on different queries: dense search tends to miss exact identifiers such as error codes, ticket numbers and product names, while BM25 misses paraphrases and synonyms. Fusing the two ranked lists recovers results that either path alone would drop, and rank fusion has been shown to outperform the individual rankers it combines (Cormack et al., 2009). But it creates a security problem: you now have two independent retrieval paths, each of which must be individually secured.

Dense Path: Filter at the Database

For dense retrieval, we pass the role filter computed by allowed_role_filter() as a metadata filter to the vector store query. This means the filtering happens at the vector database level, in the same operation as the similarity search. Only chunks whose stored role metadata intersects with the allowed roles are returned.

This is the correct place to filter for dense retrieval because:

  • It avoids loading unauthorized vectors into Python memory at all.
  • It allows the vector index to optimize the query — many vector databases (Qdrant, Weaviate, Pinecone) have native support for pre-filtering on metadata before ANN search.
  • The filter logic is expressed once, in the query, not scattered across post-processing code.

Sparse Path: Post-Filter After BM25

BM25 indices do not natively support metadata filtering in the same way. The BM25 search returns matches based on term frequency and inverse document frequency, without awareness of chunk metadata. We therefore apply visible_to() as a post-filter on the BM25 results.

This is where the explicit visible_to() function earns its keep. It is pure Python, has no side effects, and runs in microseconds per chunk. We apply it to every BM25 hit before those hits enter the fusion step.

Why We Filter Before Fusion, Not After

Reciprocal Rank Fusion computes a combined score for each document based on its rank in each individual list:

Formule mathématique de la Reciprocal Rank Fusion (RRF) avec constante de lissage k=60

Where k=60 is a smoothing constant (the value proposed by Cormack et al., 2009) that reduces the influence of very high-ranked results relative to lower-ranked ones. This makes the fusion more balanced and less susceptible to a single dominant result.

If we ran fusion on unfiltered results and then discarded unauthorized chunks, two problems arise:

  • An unauthorized chunk ranked first in the dense list would push every authorized chunk below it down one rank in that list, lowering their fused scores relative to chunks from the other list — a subtle but real influence on which authorized chunks make the final top-k (see the worked example below).
  • Filtered-after-fusion outputs an arbitrary number of remaining chunks rather than a clean top-k of authorized ones.

Filtering before fusion means the fused ranking is computed exclusively over chunks the user is authorized to see. The final top-5 is a top-5 of genuinely accessible content.

A worked example (k = 60, top-2). Suppose a support user sends a query for which the unfiltered dense list is [U1, U2, D] and the sparse list is [E, B]. U1 and U2 are tagged finance; D, E and B are all visible to this user.

Filtering before fusion, the dense list becomes [D]. D and E each score 1/61 ≈ 0.0164 and B scores 1/62 ≈ 0.0161, so the top-2 is D and E. D and E tie exactly at 1/61. The tie does not change which chunks make the top-2, since B is strictly below both, only the order in which they reach the context. A production implementation should still break ties deterministically (for example by raw dense similarity, then by chunk ID) so that the same query always yields the same context order. 

Fusing first and filtering afterwards, D sits at dense rank 3 and scores only 1/63 ≈ 0.0159, which is now below B. Once U1 and U2 are discarded, the top-2 is E and B. D, the most semantically relevant document this user is allowed to see, has been pushed out of the context by two chunks the user could never read. Nothing unauthorized reaches the model, yet unauthorized content still decided what the model gets to see. With a realistic top-k and hundreds of candidates the effect is harder to spot, which is exactly why it should be ruled out structurally.

A note on reranking 

Our pipeline does not include a reranking stage between fusion and LLM context assembly. This is a deliberate choice.

Cross-encoder rerankers — which evaluate query-chunk pairs for relevance — are increasingly common in production RAG systems and can significantly improve answer quality. However, they introduce a security constraint: the reranker must receive only pre-filtered, authorized chunks. Applying a reranker before RBAC enforcement would create a new leakage vector functionally identical to the post-fusion filtering problem described above — an unauthorized chunk could influence the ranking of authorized content before being discarded.

If you add a reranker to this pipeline, position it after fusion and after RBAC filtering, not before.

6. The Full Request Lifecycle

End-to-end, a chat request travels through six distinct systems before an answer is returned. Understanding this sequence is essential for reasoning about where failures can occur and how to trace them.

Diagramme du cycle de vie d'une requête RAG illustrant les 6 étapes de sécurité, incluant l'authentification JWT, la résolution des rôles et le double filtrage RBAC

A few notes on failure modes at each stage:

  • Stage 1 — Token validation failure. If the JWT is invalid or expired, the request is rejected before any role resolution occurs. No data is accessed.
  • Stage 2 — Database unavailable. If the role store times out (2-second hard limit), the request fails with a 503. The FALLBACK_ROLE = public logic is applied only when a role string is unrecognized, not when the database is unreachable. An unavailable role store means no access, not degraded access.
  • Stage 3–5 — Filter computation error. If allowed_role_filter() raises an exception, the request fails closed. We never fall back to an unfiltered query.
  • Stage 6 — Empty authorized chunk set. If the authorized top-k is empty (the user has no role overlap with any relevant chunk), the LLM receives an empty context and is instructed to respond with a standard "no information available" message. We do not allow the model to answer from training data when the context is empty, as this bypasses the RAG security boundary entirely.

7. Lessons Learned and What Generalizes to Other Stacks

After building and operating this system, we distilled the following principles.

What Generalizes to Any RAG System

Fail ingestion early, fail loudly. The document registry pattern — mandatory pre-registration before ingestion, hard failure for unknown documents — is the highest-leverage decision we made. It converts a runtime security property into a build-time invariant. Any RAG system handling sensitive documents should adopt this pattern, regardless of what vector store or framework they use.

How the registry works in practice. Every source document must be registered before the ingestion pipeline will touch it. A registry entry is small: a stable document ID, the source location, the owner responsible for its classification, the set of role tags it carries, and who classified it and when. The ingestion job looks up each incoming document by ID. If there is no entry, the job fails for that document instead of indexing it untagged. If there is one, its role tags are validated against the same role set enforced by the CHECK constraint on user_roles, and every chunk produced from the document inherits those tags, both in its vector-store metadata and in the BM25 index.

Classification is hybrid rather than purely manual or purely automated. Role tags are first proposed from where the document comes from: a finance SharePoint site maps to finance, an HR folder maps to hr, the public knowledge base maps to public. The document owner then confirms or corrects the proposal before the entry becomes active. Anything the mapping cannot resolve is never guessed; it stays unregistered, and therefore unindexed, until someone classifies it. When a document’s classification changes, its registry entry is updated and the document is re-ingested, so its chunks never keep stale tags.

Every retrieval path needs its own enforcement. If you adopt hybrid retrieval (and for most enterprise corpora you should, because lexical and semantic search fail on different queries), you must filter both paths independently, before fusion. Filtering after fusion does not protect ranking integrity. This is a structural requirement, not an implementation detail.

Treat public as a first-class role, not a special case. Making public an explicit role that every user implicitly holds simplifies every piece of access logic. There is no special handling, no conditional branching, no if not roles: return all_public_chunks. The same intersection logic works for every user.

The Security/Answer Quality Tradeoff

RBAC filtering improves security and degrades retrieval coverage — these two effects are inseparable. A user with a narrow role set will see fewer chunks, and some queries will return a small or low-relevance authorized corpus.

The failure mode to avoid is silent degradation: a nearly empty authorized context passed to the LLM, which then generates a confident-sounding answer from its training data rather than the retrieved content. This bypasses the RAG security boundary entirely.

Our approach:

  • Set a minimum relevance score threshold on the authorized top-k. If fewer than two chunks exceed the threshold, return a standard message: "I don't have enough authorized information to answer this question."
  • Never pass an empty context to the LLM. An empty context must produce an explicit refusal, not a fallback to model knowledge.
  • Surface the "no results" case as a first-class event in your audit logs. It is often a signal that content classification is incomplete, not that the user's query is wrong.

The right tradeoff between security and answer quality is a business decision. The architecture should make that tradeoff explicit and observable — not hide it behind a model that generates plausible-sounding answers regardless of context.

Making the tradeoff observable. In practice this means tracking three numbers per role and reviewing them with content owners, not only with engineers. The refusal rate is the share of queries that ended in the “not enough authorized information” message. The RBAC discard ratio is the share of relevant retrieved candidates that were removed by role filtering. The context relevance score is the average relevance of the chunks that did reach the model. All three come straight from the audit events described above, and they fit on a single dashboard broken down by role and by week.

These numbers turn an abstract tradeoff into a concrete decision. Suppose the support role shows a refusal rate several times higher than any other role, and the discard log shows that most of the removed chunks are engineering troubleshooting guides. That is not a bug to fix in the retrieval code; it is a question for the business. The options are to reclassify those guides so support can see them, to publish a redacted version for support, or to accept that support agents escalate those questions to engineering. Whichever option is chosen, its effect shows up in the same metrics the following week, so the decision can be checked rather than assumed.

Multi-Tenant Deployments

The architecture described in this article assumes a single-organization deployment. Multi-tenant systems — where multiple clients or business units share the same vector index — require an additional isolation layer.

Role filtering alone is not sufficient to prevent cross-tenant data leakage. A role named engineering in Organization A should not grant access to engineering-tagged chunks belonging to Organization B.

The required addition is a tenant_id namespace, enforced at the vector store level independently of and prior to role filtering:

# Example: Qdrant filter combining tenant isolation and role RBAC
{
    "must": [
        {"key": "tenant_id", "match": {"value": user.tenant_id}},
        {"key": "role",      "match": {"any": allowed_role_filter(user.roles)}}
    ]
}

Both conditions are required. Tenant isolation without RBAC allows any user of a tenant to see all of that tenant's content. RBAC without tenant isolation risks cross-tenant leakage if role names overlap. The two controls are complementary, not interchangeable.

The underlying principle is not about our stack. RBAC in RAG is an architectural commitment, not a feature you add at the end. Every layer of the pipeline — ingestion, classification, storage, retrieval, fusion, serving — must treat the role tag as a first-class property of every chunk. When you do that, security becomes a structural guarantee rather than a runtime hope.

Where to go next. This article covered who can see which chunks. For the other half of the picture (prompt injection, jailbreaks and output manipulation), read the companion article, Building Defense-in-Depth for Enterprise RAG. And if you are designing or auditing access control for your own RAG platform and want a second pair of eyes on the architecture, get in touch with our team: we are happy to review your ingestion, retrieval and fusion pipeline against the five enforcement points described here.

8. References

Cormack, Gordon V., Charles L. A. Clarke, and Stefan Büttcher. “Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods.” Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2009, pp. 758–759.

Lewis, Patrick, et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020.

Sandhu, Ravi, David Ferraiolo, and Richard Kuhn. “The NIST Model for Role-Based Access Control: Towards a Unified Standard.” Proceedings of the Fifth ACM Workshop on Role-Based Access Control, 2000, pp. 47–63.

Saltzer, Jerome H., and Michael D. Schroeder. “The Protection of Information in Computer Systems.” Proceedings of the IEEE, vol. 63, no. 9, 1975, pp. 1278–1308.

Malkov, Yu. A., and D. A. Yashunin. “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs.” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 4, 2020, pp. 824–836.

Jones, Michael B., John Bradley, and Nat Sakimura. “JSON Web Token (JWT).” RFC 7519, Internet Engineering Task Force, May 2015.

OWASP Foundation. “LLM08:2025 Vector and Embedding Weaknesses.” OWASP Top 10 for LLM Applications 2025, OWASP GenAI Security Project, 2024.

Haji AbdelilahWritten By Abdelilah HAJIFull-Stack & Data DeveloperXelops Technology

PLUS D'ARTICLES