AI Agents & Automation in Practice
Internal AI Assistant: How to Avoid Leaking the Wrong Data
By Mudabbir Hasan, Co-founder · Published · Last updated
Do not rely on the AI model or its prompt to keep data private. Enforce each user's permissions in the retrieval layer, so the assistant never receives a document that person could not open directly. Partition the data by user group, log every retrieval, validate documents before they enter the knowledge base, and test with two users of different access before launch.
Why do internal AI assistants leak data?
An assistant connected to company documents answers from whatever it retrieves. If retrieval ignores who is asking, the assistant can summarise a payroll file for someone who could never open it. OWASP's 2025 list of risks for LLM applications names this directly: LLM02, Sensitive Information Disclosure, covers confidential business data appearing in responses, and LLM08, Vector and Embedding Weaknesses, covers poorly designed permissions on the stores that hold retrieved content.
Why is the prompt not a safe place to enforce permissions?
OWASP says system-prompt restrictions offer some defence, but warns they may not always be honoured and can be circumvented through prompt injection. A rule written in the prompt is a request to the model, not an access control. The control has to sit outside the model, in the system that decides what content the model is allowed to see.
Where the permission check belongs
- 1
The user asks a question
- 2
Retrieval checks that user's permissions
Not the prompt: it can be talked around.
- 3
Only permitted documents reach the model
- 4
The answer is returned
- 5
Every retrieval is logged
What does OWASP recommend instead?
- Permission-aware vector and embedding stores, with fine-grained access controls and the data partitioned by user class or group, so one group's content cannot surface in answers for another.
- Least privilege: limit what the model can reach, including external data sources, and secure the runtime that orchestrates the data.
- Data validation and source authentication: audit the knowledge base regularly and accept data only from verified sources.
- Data classification: tag data so access levels match the people who ask.
- Detailed immutable logs of retrieval activity, so suspicious behaviour can be found.
OWASP mitigations for retrieval systems
Permission-aware vector stores
Data partitioned by user group
Least privilege for the model
Validate and authenticate data sources
Classify data to match access levels
Immutable retrieval logs
What does a poisoned document look like?
OWASP's example is a resume with hidden white-on-white text telling the model to recommend the candidate. A screening tool that ingested it followed the instruction. The same trick could be used on an internal wiki page. The fix is to extract text in a way that detects hidden content and to validate documents before they enter the knowledge base, so treat anything retrieved as untrusted input to the model.
How do you test it before launch?
- Create two test users with different access. Ask the assistant, as the lower-access user, for something only the higher-access user may see. It must not answer.
- Plant a document containing a hidden instruction in the knowledge base and check that the assistant ignores it.
- Ask the assistant to reveal its instructions and the names of documents it cannot show you, and check the answer.
- Review the retrieval log for the test session and confirm every retrieval was made with the asking user's permissions.
- Repeat the tests whenever you add a new data source.
The two-user test
Create two test users with different access
As the lower-access user, ask for the higher user's data
It must refuse.
Plant a document with a hidden instruction
Review the retrieval log for the session
Repeat for every new data source
How does this relate to our own work?
Our workforce management case pairs an internal HR and operations assistant with role-based access throughout, where a manager sees their team's data and not the whole organisation's by default. The case page does not describe how the assistant itself is built, and the engagement is confidential, so this article is general guidance and is not a description of that system.
What should you do before you connect an assistant to company data?
- List the data sources and who may see each one.
- Decide where permissions are enforced and check it is not the prompt.
- Decide who reviews the retrieval logs and how often.
- Run the two-user test before any real data goes in.
We cover the related question of when an agent should hand a case to a person in a separate guide.
Sources
- LLM02:2025 Sensitive Information Disclosure, OWASP GenAI Security Project (accessed 2026-10-10)
- LLM08:2025 Vector and Embedding Weaknesses, OWASP GenAI Security Project (accessed 2026-10-10)
Related guides
Talk to an engineer about your project
Start with a free project call. We learn what you are building and recommend the right next step, with no pitch.
Book a call