Connecting AI to your internal data safely

How retrieval works, and why permissions are the hard part

Retrieval, often called retrieval-augmented generation or RAG, works in two stages. Ahead of time, documents and records are split into passages and indexed, usually with both a keyword index and a vector index that matches on meaning. When someone asks a question, the system finds the most relevant passages and sends them to the model along with the question and instructions to answer only from them, with citations.

The model has no idea who is asking. Whatever passages it receives, it will use. So if the index was built by a service account that can read everything, and the search isn't filtered per user, the assistant becomes a friendly front end to the payroll folder. Telling the model "don't reveal HR data" in its instructions isn't a control; instructions can be ignored or talked around. The only reliable place to enforce access is before retrieval results reach the model.

Built-in assistants or a custom build

If your documents live in a platform that offers its own assistant, such as Microsoft 365 with Copilot, that may be the simplest route for general questions over documents. Microsoft states that Copilot only surfaces organizational data the signed-in user already has at least view permission for, which is exactly why cleaning up overshared sites comes first. Check current licensing, features and admin controls with the vendor.

Carry permissions through to the user

There are a few established ways to make retrieval respect each user's access. They can be combined, and the right one depends on the source.

• Identity comes from sign-in, not from the prompt. The retrieval service takes the user from the validated SSO token, never from anything typed into the chat.
• Deny by default. A passage with no access list, or one that failed to sync, is excluded rather than treated as public.
• Check again at the source when it matters. For sensitive collections, confirm the user can still open the cited document before showing the answer.

Our guide to securing internal apps with single sign-on and least privilege covers group design and access tokens, and putting AI agents to work on internal processes covers narrow tools for when an assistant needs to look up records rather than read documents.

Decide what to index, and what not to

It's tempting to connect everything and let search sort it out. A smaller, curated start is safer and usually gives better answers, because the assistant isn't choosing between five versions of the same procedure.

• Give each collection an owner who decides what's in it and keeps it current.
• Use the labels you have. If documents carry sensitivity labels or classification metadata, use them to exclude content automatically at ingestion.
• Scan before indexing. Check for passwords, keys and personal data in files headed for the index, and hold back anything that matches for review.
• Index structured data carefully. For database records, it can be better to look up live values through a narrow API at question time than to copy rows into an index where they go stale.

Stale and conflicting data

An index is a copy, and copies drift. A superseded procedure that stays in the index will be quoted with the same confidence as the current one, and a document someone deleted or restricted yesterday may still be searchable today.

• Sync changes, deletions and permissions. Use the source's change feed where it has one, such as delta queries in Microsoft Graph, and agree a maximum lag. Permission changes and deletions should propagate at least as fast as new content.
• Prefer the approved version. Index published versions, not drafts; drop or clearly tag archives; and record each document's effective date and status as metadata.
• Rank by currency where it matters. When two passages conflict, filter or rank on status and date in code rather than relying on the model to notice.
• Show the date. Every citation includes the document's date or revision, so readers can spot an old source.
• Review what gets cited. Reports of the most-cited documents show owners which content the business actually relies on, and which needs updating.

Citations people can check

Citations turn an answer from something to trust into something to verify. They also make errors visible quickly, because a reader can see when a cited passage doesn't say what the answer claims.

Least privilege for the pipeline itself

The ingestion and search services hold a copy of a lot of company data, so they need the same care as any other system that does.

• Read-only, narrowly scoped connectors. The ingestion identity can read only the sites, folders or views being indexed. On Microsoft 365, permissions such as Graph's Sites.Selected allow access to specific sites rather than all of them.
• Database logins limited to views. A read-only login with access to specific views, not the tables behind them.
• Treat the index as sensitive as its sources. Passages and their vector representations are derived from your documents; encrypt the store, restrict access to it, and include it in backups and retention decisions.
• Protect the logs. Questions and retrieved passages can contain sensitive data. Log them for audit and quality, with access limited to named roles.
• Treat indexed content as untrusted input. A document can contain text written to manipulate the model. The assistant should never gain permissions or take actions because of something it read.
• Confirm provider terms. Check what the model provider may retain from prompts that now include your documents, under your account's current terms.

A reference design

Test access, not just answers

Answer quality is tested like any AI workflow, with real questions and expected sources. Access needs its own tests, run before launch and after every change to connectors, groups or the index.

• Test as different people. Use test accounts in different groups, plants and roles, and confirm each sees only the sources it should.
• Plant marker documents. Put a restricted test document containing a unique phrase in each sensitive area, then ask about that phrase as users who shouldn't see it. It should never appear in an answer or a citation.
• Test removals. Remove a test user from a group, or delete a marker document, and measure how long until the assistant stops using it.
• Test hostile content. Index a test document containing instructions aimed at the model and confirm they have no effect.

Our guide to scoping a first AI pilot covers building an evaluation set from real history and writing acceptance criteria before you build.

A one-page checklist

Our experience includes document and email processing, classification, summarization, automated workflows and AI agents, with integrations across the Anthropic Claude, OpenAI, Google Gemini and xAI Grok APIs, along with OAuth and OIDC authentication flows and database design across SQL Server, PostgreSQL and NoSQL stores. Before rolling an assistant out to staff, agree an AI acceptable-use policy that sets the rules for what data goes where. See AI integration for how we'd approach connecting AI to your data.

Sources

• Data, privacy and security for Microsoft Copilot (Microsoft)
• Use delta query to track changes in Microsoft Graph data (Microsoft)
• Overview of Selected permissions in OneDrive and SharePoint (Microsoft)
• LLM01: Prompt injection (OWASP GenAI Security Project)