What to ask an AI vendor about permissions

Most AI evaluations spend their time on the wrong risk. The demo gets scrutinised, the answers get graded, someone asks about hallucination rates — and the thing that actually determines whether this software can be switched on for four hundred people rather than four never comes up.

That thing is the permission model. It is also the least fun question to ask, which is why it is the one worth having written down before the call.

What follows is product-neutral. It applies to an assistant bolted onto a tool you already own, to a platform that claims AI throughout, and to anything in between.

Why this is the question that decides the rollout

An AI feature that can see more than its user is not a feature with a security caveat. It is a data-access change wearing a chat interface.

The pattern repeats: the pilot goes well because the pilot group is ten people in one department who can all see each other's data anyway. Then the rollout reaches finance, or HR, or a regulated team, and someone asks what the assistant can read. If the answer is "everything in the index", the project stops there — after the budget, the integration and the change-management effort have all been spent.

Asking these eight questions in the first demo costs twenty minutes and tells you whether that wall exists.

1. Whose permissions does the AI use?

Ask: when the assistant reads a record, whose access is it using — mine, or its own?

The answer you want: yours. The AI runs as the calling user, inside the same roles and record-level rules, and cannot retrieve anything you could not open yourself.

The answer you will often get: a dedicated service account with broad read access, plus an assurance that the system prompt constrains it. That is not an access control. It moves the authority question out of the role model — where it is reviewed, audited and offboarded — and into a prompt, where none of those things happen.

2. What happens when it is asked for something out of scope?

Ask: what does the assistant do when I ask it about a record I am not allowed to see?

The answer you want: the same thing the application does — it is refused, by the same code path, and the refusal is indistinguishable from the record not existing.

Listen for: "it will tell you it can't help with that." A model declining is a behaviour. A system refusing is a control. If the enforcement lives in the model's judgement rather than in the query, then a differently-worded question is a bypass.

3. Where does retrieval get its permissions?

This is the one that gets missed, and it is the one that has caused the most quiet incidents.

Ask: when the AI searches across our content, is that index built with my permissions or with elevated ones — and when a document's permissions change, when does the index find out?

The answer you want: retrieval is filtered by the asker's access at query time, and permission changes propagate promptly.

Why it matters: an index built once by a privileged crawler is a permanent, searchable copy of everything, sitting behind a natural-language interface that is very good at summarising. The document you revoked access to last month can still be described to someone who was never allowed to open it — accurately, in a paragraph, with no access log entry against the file itself.

4. Can the AI widen its own access?

Ask: can the assistant perform an action that changes permissions — adding someone to a group, sharing a document, changing a role?

The answer you want: either it cannot, or those actions are a separately governed class requiring explicit approval.

Why: an agent that can act on your behalf and can also grant access has, in effect, unbounded access one step removed. This is worth asking even where the answer is obviously no, because the answer tells you whether anyone has thought about action classes at all.

5. What does the log say it did?

Ask: show me the audit trail immediately after the assistant does something.

The answer you want: the AI's action sits in the same log as human actions, identifying both the AI and the person it acted for, with a before and after.

The tell: a separate "AI activity" screen. Two logs mean that reconstructing what happened to a record requires correlating them by timestamp — which is exactly the reconstruction you will be attempting, under pressure, on the day it matters.

6. Who can turn it on, and for whom?

Ask: who controls whether this is enabled, at what granularity, and can it be turned off for one team without turning it off for everyone?

The answer you want: an administrator, per role or per group, with the off switch as easy as the on switch.

This is the question that decides whether a legal or clinical or payroll team can be carved out — and whether you can turn it off in an incident without a support ticket.

7. What leaves the building?

Ask: which providers process our content, is it used for training, and where is it processed?

The answer you want: a named list of sub-processors you can put in front of your own compliance people, a contractual no on training, and a straight answer on region.

"We use a leading AI provider" is not an answer. If the vendor cannot name it, they cannot tell you when it changes either.

8. What happens when someone leaves?

Ask: when I offboard a user, what happens to what the assistant knew on their behalf — saved threads, generated summaries, anything cached?

The answer you want: the same lifecycle as their other data, under the same retention policy, deleted by the same process.

Why it is worth asking last: it is the question most likely to be met with a pause, and the pause is the information. Offboarding is where a bolted-on feature reveals that it has its own store nobody has mapped.

The short version

If you want one paragraph in an RFP:

Describe how the AI feature's access is derived from the requesting user's permissions, including for retrieval and search. State whether any component operates with elevated or service-account access, and if so, what constrains it. Confirm that AI actions are recorded in the same audit trail as user actions. Name all sub-processors that receive customer content, and confirm that content is not used for model training.

Four sentences. A vendor who can answer them in writing has thought about this; a vendor who cannot has not, and the pilot is where you will find out.

What a good answer sounds like overall

You are listening for one property across all eight: the AI is a caller, not an exception. It uses the same permission model, writes to the same log, is governed by the same admin controls and is subject to the same retention rules as a person.

Every weak answer is a variation of the same thing — a second path, with its own account, its own store, its own log or its own rules. Second paths are not automatically wrong, but each one is a thing to review, a thing to offboard and a thing that will be forgotten. Counting them is a decent proxy for how much work this will be to own.

Common questions

Is this only relevant for regulated industries?

No. Regulated buyers ask these questions earliest, but the failure mode — an assistant summarising a document the asker could not open — is equally awkward at a forty-person company, where the document is more likely to be someone's salary than a patient record.

The vendor says the model does not retain our data. Is that enough?

It answers one question of the eight, and not the one that decides the rollout. Retention is about what happens to data after it is sent. Permissions are about what gets sent in the first place.

What if the AI is read-only today?

Ask anyway, and ask what the roadmap says. Read-only is a temporary property of most AI features, and the permission architecture is decided long before the write path is added — usually by whoever was in a hurry.

Which one question would you ask if you only had one?

The third. Retrieval permissions are the most commonly wrong, the least commonly asked about, and the hardest to retrofit, because fixing them means rebuilding the index rather than changing a setting.

---

If you would like to be among the first to use Cyril, join the waitlist.