What is Secure Document Retrieval in Enterprise RAG?
Secure document retrieval means a RAG system only retrieves the documents and chunks the requesting user is authorized to see, and that rule is enforced in the retrieval layer rather than left to the LLM. In practice it combines authorization-aware filtering, an appropriate tenant isolation model, timely propagation of permission and deletion changes, isolated caches, fail-closed behavior, prompt injection defenses and monitoring of every retrieval.
RAG is moving from pilots into production, and that changes the risk. Grounding a Large Language Model (LLM) in your internal knowledge base reduces hallucination, but it also gives the model a path to data the user may not be cleared to see. If the retriever pulls a board paper to answer a junior employee’s question, the LLM will happily summarize it, and your data silos are gone.
That is why retrieval has to be treated as an access control decision, not just a similarity search. The core controls are pre-retrieval authorization filtering, filtered search that stays accurate without weakening the filter, a tenant isolation model that matches the risk, and defenses against indirect prompt injection (IPI) and retrieval manipulation. Around them sit the operational controls that are easy to miss: permission and deletion propagation, cache isolation, fail-closed retrieval and retrieval observability. Confidential computing adds hardware protection for data in use where the threat model calls for it.
Why Post-Filtering Fails
The simplest approach is post-filtering: run the semantic search across the whole corpus, take the top 20 results, and let application code drop the chunks the user is not allowed to see.
This breaks down in production for two reasons:
- Recall starvation: If 19 of the top 20 results are restricted, the filter removes 19 and the LLM answers from a single chunk. Accuracy drops, and nothing in the logs tells you retrieval failed.
- Side channels: The time it takes to filter varies with how many restricted documents matched. An attacker watching response times can infer that sensitive documents exist and roughly what they are about.
The alternative is pre-retrieval filtering. The requesting user’s identity and authorization context should be enforced during retrieval. Depending on the enterprise architecture, this may use RBAC, ABAC, ACLs, tenant boundaries, source-level permissions or other policy-based authorization mechanisms. Authorization metadata should be preserved at chunk level so restricted content is excluded before it reaches the model. The authorization context should come from the authenticated session or a trusted policy service, never from the user’s prompt or from the LLM itself.
Filtered Vector Search and HNSW
Most vector databases use Hierarchical Navigable Small World (HNSW) graphs, or similar graph indexes, for fast approximate nearest neighbor (ANN) search. Filtering inside that search does not weaken access control: when the filter is enforced as a hard condition, restricted chunks are not returned. What filtering can affect is recall and latency, meaning whether the search finds the best authorized chunks, and how quickly.
How much depends on the engine and on how selective the filter is. Some engines traverse the full graph and simply skip non-matching nodes when collecting results, which keeps the graph connected but wastes work when few nodes match. Others restrict traversal to matching nodes, and that is where disconnection appears: under a highly selective filter, such as a small tenant or a narrow clearance level, permitted nodes can form islands the search cannot cross, so it stops before reaching the true nearest neighbors. Engines that apply the filter after the ANN step can return fewer than the requested number of results, and many query planners switch to exact or index-based search when a filter matches only a small share of the data.
Filter-aware techniques reduce the recall problem. ACORN (Patel et al., SIGMOD 2024) searches the subgraph of matching nodes using denser neighbor lists or two-hop expansion, so traversal can reach permitted nodes that are not directly linked [1]. Weaviate offers an ACORN-based filter strategy from version 1.27 [2], and Qdrant adds extra graph links for indexed payload values and falls back to a payload index scan when filter cardinality is low [3].
For security, two rules follow. Treat filtered-search quality as a recall and performance issue to test with realistic permission distributions, including users who can see only a small fraction of the corpus. And never add a fallback that retries without the authorization filter, or loosens it, when too few results come back.
Tenant Isolation: Choose the Boundary Deliberately
Tenant isolation should not be assumed from a single mechanism. The options range from a tenant filter in a shared index, through per-tenant namespaces, partitions or collections, to separate indexes, databases, cloud accounts and encryption keys. Each step reduces the blast radius of a mistake and adds cost and operational overhead, so the right choice depends on data sensitivity and on regulatory and contractual commitments.
A separate namespace or index is not automatically a hard security boundary. Depending on the product, a namespace may be a logical label or a separately stored partition [4], but it is often reached with the same credentials and selected by application code. It holds as a boundary only when the tenant scope is derived server-side from the authenticated identity, enforced on every query path (vector, keyword and hybrid search, reranking and caching), and backed by access controls on the store itself, such as credentials or roles scoped to that tenant.
Isolation also has to cover everything around the index: ingestion queues, embedding and reranking services, caches, conversation memory, logs, backups and evaluation datasets. And tenant isolation does not replace per-user authorization inside a tenant, where different users still see different documents. Test the isolation model you choose rather than inferring it from the architecture diagram.
Permission and Deletion Propagation
Chunk-level authorization metadata is a copy of permissions that live somewhere else, in SharePoint, Confluence, a document management system or an identity provider. When the source changes and the index does not, the RAG system keeps enforcing yesterday’s access rules.
Permission propagation: When a document’s ACL changes, a user leaves a group or content is reclassified, the change has to reach every derived chunk within a defined window. Use event-driven sync from source systems where available, with scheduled reconciliation to catch missed events, and resolve group membership, including nested groups, consistently. For highly sensitive content, check the user’s current entitlement against the source of truth at query time rather than relying only on metadata copied at ingestion.
Deletion propagation: When a source document is deleted, or personal data has to be erased, the change has to reach every derived representation: chunks, embeddings, keyword indexes, caches, generated summaries and evaluation datasets. Each chunk needs lineage metadata, such as source ID and version, so these copies can be found. Many vector engines mark deleted vectors and remove them later during compaction, so confirm when deleted content stops being retrievable and when it is physically removed. Backups and legal retention requirements need their own documented handling.
Test both paths end to end: change a permission or delete a document at the source, then confirm it stops being retrievable within the agreed window.
Cache Isolation and Fail-Closed Retrieval
Cache isolation: Response caches, semantic caches and retrieval-result caches can serve one user’s answer to another. Semantic caches carry extra risk because they match similar questions, not just identical ones. Cache keys should include the authorization context (user or role scope, tenant and policy version), cached entries should carry the most restrictive permissions of the chunks they were built from, and caches must be invalidated when permissions change or content is deleted.
Fail-closed retrieval: If the identity context is missing, the policy service times out, a filter cannot be built or a chunk has no authorization metadata, retrieval should return nothing rather than fall back to an unfiltered search. Deny by default, treat unlabeled chunks as restricted, and send every fail-closed event to monitoring. A clear “no authorized results” response is better than a helpful answer built from content the user should not see.
Indirect Prompt Injection (IPI)
Because RAG systems read content from sources you do not fully control, they inherit a class of attacks described in the OWASP Top 10 for LLM Applications [5].
Indirect prompt injection (IPI) is the most important of these. An attacker plants instructions in content your system will ingest, such as an inbound customer email or a compromised internal document. When an authorized user later asks a related question, the poisoned chunk is retrieved and the LLM may follow the hidden instructions.
The risk grows with agentic workflows built on frameworks like LangChain or LangGraph. If the agent can call APIs, a poisoned document can tell it to send the user’s data to an attacker’s server. Defenses include sanitizing content at ingestion, keeping system instructions separate from retrieved text, limiting which tools an agent can call, and regular adversarial testing.
Retrieval Manipulation Testing
Indirect prompt injection depends on a poisoned chunk being retrieved. Retrieval manipulation is the step before that: crafting content so it ranks highly for the questions an attacker cares about, with or without embedded instructions. The PoisonedRAG study reported a 90% attack success rate by injecting five crafted texts per target question into a knowledge base of millions of texts [6].
Test whether adversarial or manipulated documents can disproportionately influence similarity ranking, dominate top-k results or cause malicious or inappropriate content to be selected for sensitive queries. Useful cases include documents stuffed with target keywords or tuned to the embedding model, near-duplicates that crowd legitimate sources out of the top-k, content that games the reranker or the keyword side of hybrid search, and low-trust sources outranking authoritative ones for high-impact questions about payments, HR or legal matters.
Rerun these tests whenever the embedding model, chunking strategy, reranker or top-k setting changes, since each shifts ranking behavior. Mitigations include source trust weighting, provenance checks at ingestion, diversity limits on top-k results and alerts on sudden ranking shifts. The same test plan should cover the controls above: permission changes, deletions, cache reuse across users and fail-closed behavior when the policy service is unavailable.
Confidential Computing
Software access controls assume the infrastructure operator and host can be trusted. For organizations handling Protected Health Information (PHI), regulated financial data or proprietary models, that assumption may not be acceptable. Similarity search and inference need data in the clear in memory, which leaves embeddings, document text and model weights exposed to a compromised hypervisor, a malicious insider with host access or memory scraping.
Confidential computing addresses this by running workloads inside hardware Trusted Execution Environments (TEEs) and releasing keys only to environments that can prove what they are running. For a RAG system, the architecture matters as much as the technology:
- Put every component that handles plaintext inside the boundary. That includes the ingestion and embedding services, retrieval orchestrator, reranker, vector database and LLM inference, not just the database and the model. Any component outside the TEE that sees chunks or prompts in the clear becomes the weak point.
- Match the TEE to the workload. CPU services can run in confidential virtual machines based on AMD SEV-SNP or Intel TDX, which encrypt VM memory with keys the host cannot read. GPU inference on NVIDIA H100 or newer GPUs in confidential computing mode runs together with a CPU confidential VM: data crossing the PCIe bus is encrypted, while on-package GPU memory is protected by isolation rather than encryption [7]. AWS Nitro Enclaves isolate CPU and memory from the parent EC2 instance but have no persistent storage or external networking and communicate only with the parent instance, so they suit narrower tasks such as key handling, tokenization or PII redaction rather than a full vector database or LLM [8].
- Protect data between components. Use attested TLS or an equivalent mechanism so each service verifies the other’s attestation before exchanging chunks, prompts or embeddings, and encrypt the vector store’s persistent data with keys that are released only to attested instances.
- Bind key release to attestation. A key broker or KMS should release keys for model weights, indexes and documents only after verifying CPU and GPU attestation evidence against an expected measurement policy. That policy has to be updated through change control whenever the software stack changes.
Confidential computing narrows who can read data in use. It does not enforce retrieval authorization, stop prompt injection or protect data once it leaves the boundary in responses, logs, telemetry or caches. Its guarantees also depend on each vendor’s threat model: TEE.fail (2025) showed that an attacker with physical access to a server’s DDR5 memory bus could extract secrets, including attestation keys, from Intel and AMD TEEs, and both vendors treat such physical attacks as out of scope [9]. Treat confidential computing as a risk-based hardening layer for the most sensitive workloads, alongside the retrieval controls above rather than instead of them.
Retrieval Observability
Retrieval decisions should be as auditable as any other access decision. For every RAG interaction, record the requesting identity, the query, the retrieved document and chunk IDs, the authorization decision and policy version applied, the model and embedding versions, the final response and any downstream tool actions, and stream these events to your SIEM.
Use the logs for detection as well as investigation. Alert on unusual retrieval volumes, repeated probing for restricted topics, spikes in authorization denials or fail-closed events, unexpected index or permission changes, cache hits that cross user or tenant boundaries, prompt-injection indicators and abnormal agent tool activity.
These logs contain personal data too, so give them their own retention limits and access controls, and include them in deletion workflows.
Compliance and Governance: DPDP Act, GDPR and SOC 2
Regulators expect data minimization, and a RAG system should apply it at query time by retrieving the fewest chunks needed to answer the question.
Under the DPDP Act and GDPR, it also makes sense to detect and redact or pseudonymize personal data before it is embedded, using tools such as Microsoft Presidio [10].
The retrieval logs described above support SOC 2 Type II monitoring controls and speed up incident response, while permission and deletion propagation provide evidence that access changes and erasure decisions actually reach the RAG index.
How ServQual and SUSAN Help
Securing retrieval in AI systems takes good architecture up front and continuous checking afterwards.
- Secure retrieval architecture: We help security teams design retrieval layers that enforce the organization’s authorization model and an appropriate tenant isolation model at the data layer, so the LLM cannot widen its own search scope.
- AI threat modeling and Privacy by Design: We map AI data pipelines against the OWASP Top 10 for LLM Applications and MITRE ATLAS, and advise on confidential computing and pseudonymization to reduce the risk of knowledge base poisoning and IPI.
- Continuous assurance with SUSAN: SUSAN gives CISOs and GRC teams a live view of their AI-related controls against HIPAA, GDPR and SOC 2, including audit logging and identity controls.
“In enterprise RAG, retrieval is an access-control decision, not just a similarity search. If authorization fails before retrieval, the model should receive nothing.”
Shubham Choudhari
Security Engineer | ServQual
FAQ
Most frequent questions and answers
Post-filtering searches across the full corpus first and removes restricted results afterwards. This can reduce answer quality and create timing side channels that reveal the existence of sensitive documents.
Pre-retrieval authorization enforces the requesting user’s identity and permissions during retrieval itself, using mechanisms such as RBAC, ABAC, ACLs, tenant boundaries or source-level permissions.
When authorization filters are enforced as hard conditions, restricted chunks are not returned. The main impact is on recall and latency, especially when highly selective filters make authorized data harder to retrieve efficiently.
Tenant isolation separates data and retrieval scope between tenants using filters, namespaces, partitions, indexes, databases, cloud accounts or encryption keys. The boundary should be enforced server-side and tested across every retrieval path.
Fail-closed retrieval means returning no results when authorization context is missing, a policy service is unavailable or a chunk lacks authorization metadata, rather than falling back to unrestricted search.
Indirect Prompt Injection occurs when an attacker hides malicious instructions inside content that later enters the RAG knowledge base. When that content is retrieved, the LLM may follow those instructions and trigger unauthorized actions or data leakage.
Permission changes and deletions should propagate from the source system to all derived chunks, embeddings, indexes, caches and generated artifacts within a defined window, with lineage metadata used to locate affected content.
The article positions SUSAN as supporting continuous assurance for AI-related controls, including audit logging, identity controls and compliance visibility, while ServQual supports secure retrieval architecture and AI threat modeling.
Secure Your Enterprise RAG Retrieval Layer
Assess authorization, tenant isolation, vector database controls, Prompt Injection exposure, cache isolation and retrieval observability with ServQual.
Disclaimer: This article is provided for general Cybersecurity and RAG Security guidance. Appropriate controls depend on the architecture, data sensitivity, authorization model, threat model and regulatory context of each environment.
