The seven-question test for whether AI is really built into a platform

Most buyers currently evaluate AI the way they evaluate a feature: they watch it work once. That is exactly the wrong test, because a demo is the one environment where a bolted-on assistant performs well. The prompt is chosen, the data is clean, and the question stays inside one product.

The failures happen later, and they are structural rather than about model quality. The frontier models are excellent. What separates platforms is what the model is allowed to see, what it is allowed to do, and what happens when it is wrong.

1. Can it answer a question that crosses two modules, without an integration?

Ask for a question that spans two parts of the business at once. “Which customers are at risk this quarter, and why?” is a good one, because a real answer needs the pipeline, the support history, the delivery status and the invoices.

A strong answer is a single query against a single data model, and the vendor can describe the shape of that query.

A weak answer contains the word “integration”, or the demo silently narrows the question — you asked about risk across the business and got an answer drawn from the pipeline alone. That narrowing is the tell. It means the assistant can only see one silo, and the parts it cannot see are the parts that would have changed the answer.

2. Does every record have a representation built for a model, not for a database?

An AI grounded on raw table rows has to infer meaning from column names. That works until two tables use status to mean different things.

A strong answer is that every entity serialises itself into a deterministic, schema-aware view built specifically for model consumption — including the related state a human would treat as obvious context. In Cyril that is the ai_context serializer, which the core records have today; more record types are being added.

A weak answer is “we pass the record to the model”, or “we run retrieval over your data”. Retrieval across documents is a good complement and a poor substitute: it answers what was written down, not what is currently true.

3. Is there one permission model, and does the AI inherit it?

This is the question that decides whether AI can ever be turned on for the whole company rather than a pilot group.

A strong answer is that the AI runs as the user, under the same role and record-level permissions, and cannot see anything that user could not open themselves.

A weak answer is a separate service account with broad read access, and a promise that the prompt tells it to behave. Prompt instructions are not an access-control mechanism. If the only thing between a support agent and the payroll table is a sentence in a system prompt, the model does not have to be malicious for that to fail — it only has to be helpful.

4. Is there one audit log, and is the AI in it?

Ask to see the audit trail after the demo action, and look at where the AI’s action was recorded.

A strong answer is that it appears in the same log as human actions, with the actor identified as the AI acting on behalf of a named user, exactly like any other write.

A weak answer is a separate “AI activity” screen. Two logs mean nobody can reconstruct what happened to a record without cross-referencing them, and in the moment you most need that — a customer disputing a change, an auditor asking who approved something — reconciling two systems by timestamp is how facts get lost.

5. Can an AI action be undone?

Answering a question wrongly wastes a minute. Taking an action wrongly sends the email, moves the deal, credits the invoice.

A strong answer describes a rollback path that is a property of the platform rather than of the AI feature: soft deletes, versioned records, transactional multi-step writes — the same mechanism that lets you undo a human mistake.

A weak answer is “the user approves each step”. Approval is a good control and it is not a recovery mechanism. People approve things. The question is what happens on the day someone approves the wrong thing.

6. Can something outside the product drive it?

The real test of whether a platform’s capabilities are available to AI is whether an agent the vendor did not write can call them.

A strong answer is a documented tool surface — an MCP server or equivalent — where the actions available to an external agent are the same actions the product performs internally, under the same permissions.

A weak answer is a REST API and a shrug. An API is necessary and it is not the same thing: it exposes endpoints for a developer to assemble, not capabilities for an agent to choose between.

7. Can you change model provider without changing the product?

A strong answer is a model abstraction layer, with the current provider named as a default rather than as part of the architecture, and routing decided per task.

A weak answer is a vendor whose AI story and whose model provider are the same story. Model capability and price have moved sharply every few months for three years. A platform that needs rewriting to take advantage of that will not take advantage of it.

How to score it

Seven questions, three outcomes.

• Five or more strong answers. The AI is in the foundation. Whatever it cannot do yet is a roadmap problem, which is a normal problem.
• Two to four. A capable product with an assistant attached. It will do useful things inside one silo. Do not plan a cross-company workflow on it.
• Fewer than two. You are buying a chat window. That can still be worth having — price it accordingly.

Notice that none of the seven asks how good the answers are. Answer quality tracks the frontier and improves on its own. Architecture does not: nobody retrofits a single data model onto a product assembled from five acquisitions, because that migration costs more than the product earns. That is why these are the durable questions and “how clever is it” is not.

Cyril’s own scorecard, honestly

It would be cheap to publish a test and not sit it.

Cyril answers strongly on 1 and 7: one graph across sales, projects, support, docs and financials, and a model abstraction layer with no hardcoded provider. It answers partly on 2 and 3: the core records have an ai_context serializer, with more record types being added, and every step of an AI plan is checked against the asking person’s role on the server before it runs, though record-level ownership limits do not yet apply to every agent read.

On 4, 5 and 6 it does not answer strongly yet. Every agent run is recorded, with the plan, who approved it and what each step did, but that record sits apart from the log of people’s actions. There is no undo for an AI action yet. And an MCP connector for outside agents is in development, not available.

Two caveats the test does not cover, stated plainly: Cyril is pre-launch, and its fully autonomous Agentic mode is coming soon rather than shipped. What exists today is the platform and an AI assistant that works alongside you, showing its plan before it acts. When you run these seven questions past any vendor — this one included — hold them to the same line between what exists and what is planned.