Please Don't Say That: Stopping AI From Giving Bad Advice, Leaking Data, or Breaking the Rules

Architecture beats wishful prompting.

The most common questions we get are about AI mistakes - the expensive ones. Financial, regulatory, legal.

What stops a new AI system from giving away personal or private client data? What stops it promising something a business can't deliver? What if it produces poor financial advice or strays into services that breach FCA rules? And what happens when it says something that costs a client - or the business - serious money, especially when "the AI did it" is no defence in court?

Those risks are real. Once an AI system can see client files, private information, previous advice or proprietary data, the question is no longer whether mistakes are possible. It becomes how you stop it crossing the line: producing the wrong thing, for the wrong person, without the right controls.

This is not a capability problem. The protections in this post can all be built. What matters is how much risk a business is prepared to carry. In most cases we build low-risk models - purely because low risk equals accurate. There's also a very low-risk version. And a banking-grade zero-risk option exists, but it offers almost no practical value for most businesses and largely kills the point of adopting AI in the first place.

Four options:

• Practical structure (low risk)
• Gold-standard (very low risk)
• Hybrid (start practical, strengthen later)
• Zero-risk (switched off)

We've already built the hybrid version and prepared the gold-standard for a legal practice. Most businesses should start with the practical structure and only add more protection where it's actually needed.

Where a badly designed integration fails

It's never dramatic. The output just arrives fluent, confident and wrong. Nothing about it looks off.

A system given real company data - or left free to use its own general knowledge - goes wrong in four predictable ways.

Retrieval looks for relevance, not permission
It returns the closest match to the question. Relevance and permission are not the same thing. Unless you check entitlement at the moment of the request, it will happily surface your most sensitive material because specific almost always beats general.

Permissions sit atop of whole files; leaks happen in the cracks
Source systems control access at file or folder level. Retrieval works on fragments. Those fragments don't automatically carry the same rights. So someone can be entitled to a document and still get passages that were never meant to leave the building.

Once it's inside the model, origin disappears - and it will invent what it doesn't have
Everything in the context window becomes flat. The model can't reliably tell what it was given in confidence from what it's allowed to say, or from what it just thinks it knows. When retrieval is thin it falls back on general knowledge and answers anyway. The output has no usable record of how it was built, so you can't properly audit it or prove it was safe.

Every connection is another way out, and combining information just makes things worse
An answer can be stitched together from multiple fragments, or from retrieved data mixed with the model's own knowledge. Nothing was improperly accessed, but something was still inferred or invented and then stated as fact.

The first two problems come mostly from putting broad internal material into one retrieval pile and hoping the system is careful. Do the opposite and those two shrink significantly. The last two stay real no matter what you build.

"Just write a stronger prompt" is not a control

The usual response is to treat all of the above as a prompting problem. Firmer instructions. Guardrails. A capital-letter ridden North Star that says "Never Disclose Confidential Information".

None of that is a control. A system prompt asks. It doesn't prevent. It's just more text competing for attention. It works until it doesn't - until someone rephrases the request, or the retrieval returns something unexpected, or the model decides this case is obviously an exception.

Single layers fail. These are architecture problems.

1. The Practical Solution (low risk)

This is what works for most professional businesses.

Two classes of data

The indexed material should only be general company information: who you are, what you do, services, policies, standard terms. Nothing client-specific. Nothing personal. If the whole index was printed and left on a train, the damage would be zero.

Everything else - client records, figures, matter history - does not go into that index. It is never bulk-loaded into a second database. Sensitive data is fetched on demand for that specific request only, cleaned in memory, injected, then discarded. The AI only ever sees what it needs for that moment.

If even that is too much exposure, the AI doesn't generate the output at all. The action is triggered through your existing systems (or handed to a human) so the sensitive content never touches the model.

Entitlement at the point of request
Every request is checked against who is actually asking, right then. Most leaks die here.

Constrained generation
The model only answers from permitted material. If it can't ground the claim, or it would cross into regulated advice, it refuses.

Independent review - the Policeman
A separate agent reviews every candidate output against policy before anything leaves. It doesn't write the answer. It only passes, redacts or blocks. Because it didn't create the output, it has no interest in defending it.

Recorded human sign-off
Anything sensitive that needs to go to an approved recipient either gets logged approval or is sent directly from source via API. Content is fingerprinted so you can prove what was approved is what was sent.

Out-of-band approval and limited destinations
Sign-off happens outside the AI conversation. Every output destination is registered in advance.

All of this is built and then attacked in a Parallel environment before any real client sees output. We try to break it. Output stays internal until it holds.

2. Gold Standard (very low risk)

For the highest-risk environments the practical setup is extended:

• Infrastructure isolation
• Fully tool-mediated access only (no free-form retrieval)
• Zero AI output of sensitive material - the AI talks around the subject and triggers actions instead
• Deterministic rules in front of the LLM review
• Dual authorisation on high-risk releases
• Continuous red-teaming after go-live
• Immutable audit logs

Correct where the risk justifies the cost. For most businesses it's more than they need on day one.

3. Hybrid

Start with the practical structure. Add stronger controls only where they're needed. Design it from the beginning so you can raise the protection later without rebuilding. Most hybrid builds start practical and only tighten as real usage shows where the risk actually sits.

4. Banking-grade (zero risk)

Keep the data disconnected and the capability switched off. Complete safety. Also the least useful option for almost everyone reading this.

The only reason to list it is this: the moment you let an AI see client data, you have accepted some risk. That is usually the right commercial call. It should just be a conscious one.

What this looks like from your side of the table

You give read-only API/webhook access to the systems you already run. We identify the pinch points and define the narrowest data needed for each type of request.

Sensitive data is not stored in a second database. It's fetched for the specific query, used in memory, then discarded. General material can be indexed. Private material is not.

The most useful conversation is usually about the rules only you know: who can see what, what must never go out without review, where advice stops and regulation begins. A few real examples (good and bad) calibrate the review layer better than abstract policy.

You don't need to tidy your data first. Messy is normal.

What comes back is narrower than the demos. General questions can be answered directly. Anything sensitive either gets the minimum data for that moment, or the system triggers the action through your existing tools instead of generating the content. Nothing writes back on day one. Everything runs alongside the current process until it's clearly better and safer.

What this actually comes down to

The question isn't whether AI can see your data. It's whether you can control what it's allowed to do with it.

Most professional businesses don't need banking-grade isolation. They need separation between general and sensitive material, on-demand access instead of bulk exposure, independent review of outputs, and human approval where it still matters. That combination is achievable and proportionate.

Get the architecture right and the scary questions largely disappear.
Get it wrong and no amount of prompting will stop the mistakes.