A mid-size company rolls out an internal RAG chatbot over its shared drive, HR policies, finance reports, legal contracts, board decks, all indexed into one vector database so anyone can ask a question and get an instant answer. The demo goes well. Two weeks later, someone in a junior support role asks the bot a routine question about expense policy, and the answer comes back with a paragraph lifted straight from a document they never had permission to open, a draft compensation review. Nobody wrote code to expose that file to them. The retriever simply never checked.
Why does a RAG chatbot leak documents a user shouldn't see?
Because similarity search has no concept of who is asking. A standard RAG pipeline embeds every document into one shared vector index, and at query time it returns whichever chunks are semantically closest to the question, regardless of which user is asking or what they're cleared to see. If a compensation review is embedded in the same index as the expense policy, and the question happens to be close in vector space, it can surface, and the LLM will summarize it faithfully because summarizing faithfully is exactly what it's designed to do.
This isn't an LLM problem, it's an architecture problem. Supabase's own guide to RAG with permissions is explicit that authorization has to be enforced as a filter on the vector store query itself, before the LLM ever sees a candidate chunk, not as a disclaimer or a prompt instruction asking the model to 'only discuss what the user is allowed to see.' A prompt is a suggestion. A query filter is a constraint.
How does permission-aware retrieval actually work at the vector layer?
- Tag every chunk at ingestion time with metadata that mirrors the source system's real ACLs, tenant/org ID, department, classification level, allowed roles, synced from the actual permissions in SharePoint, Google Drive or the source CRM, never re-invented separately and left to drift out of sync.
- Apply the authorization filter before the similarity search runs, or as a hard pre-rerank filter, so a document the user can't see is excluded from the candidate set entirely rather than retrieved and then hidden after the fact.
- Start with RBAC, filtering by role, for a simple org with a handful of clean roles; move to ABAC, filtering by a combination of tenant, department and classification, once the organization has enough overlapping access patterns that role alone can't express who should see what.
- Re-validate permissions on a short TTL rather than caching them for the life of a session, since access gets revoked mid-day, an offboarded employee's session shouldn't keep retrieving documents an hour after their access was pulled.
- Log every retrieval decision, which chunks were eligible, which were actually returned, tied to the user and query, so an access review is a query against a log table, not a manual reconstruction after an incident.
What breaks when access control gets bolted on after launch instead of designed in?
Retrofitting permission filtering onto a live RAG system is expensive precisely because the mistake is structural, not superficial. If every document went into one shared index without doc-level metadata boundaries at ingestion time, there's no clean way to filter after the fact, the metadata that should have tagged each chunk's source and classification was never captured, so the fix isn't a config change, it's a full re-ingestion and re-embedding of the corpus with the correct boundaries this time.
Truto's 2026 guide to document-level RBAC in enterprise RAG pipelines makes the same point from the integration side: the ACL sync from source systems into vector metadata has to be a first-class, continuously running pipeline, not a one-time import, because source permissions change constantly and a stale metadata tag is functionally the same failure as no access control at all.
How AIBOOTSTRAPPER helps
When we built ComplySpark for ComplyNexus, a document copilot that drafts and updates compliance documents from a company's own policy library, access-aware retrieval and a full audit trail weren't add-ons, they were the reason a general counsel could trust the system enough to put it in front of a legal team. The result was documents drafted in an afternoon instead of three days, with zero off-policy language slip-ups, because retrieval was constrained to the right source of truth from day one, the same discipline covered in our guide to building a RAG pipeline for customer support.
If you're rolling out an internal knowledge assistant over documents that shouldn't all be visible to everyone who can type a question, book a call and we'll walk through what a permission-aware retrieval layer looks like for your actual document set.
Want this done for you?
Book a free strategy call and we'll show you how to build and market your business with AI.