Model selection usually gets discussed in terms of the things you would expect. Capability. Cost. Latency. Occasionally someone mentions, almost in passing, that if regulation requires certain data to stay inside your own infrastructure then a self-hosted open model may be the only option you have.
That is a whole architecture decision, buried in a subordinate clause, and for anyone building in a regulated market it is the most consequential thing about the choice.
The prompt is the transfer
Here is the thing that keeps catching teams out.
"We do not send customer data to an AI provider" is usually said in good faith. It is often true of the first version, where somebody types a question and the model answers from general knowledge.
Then the system gets useful. Retrieval is added, so the agent can answer about your customers. Tools are added, so it can look things up. And now, on every call, the prompt contains whatever the retrieval step just pulled.
The context window is not a workspace inside your building. It is the request body. Everything in it leaves, every turn.
| What people picture | What actually leaves |
|---|---|
| A question, answered generally | The system prompt, plus the user's message |
| Data stays in our systems | Whatever retrieval matched, verbatim |
| One call, one lookup | The whole accumulated history, re-sent every turn |
| The model "learns" nothing | Correct, but transmission already happened |
The context window is not a workspace. It is the request body, and it leaves every turn.
The last row matters, because it is where the reassurance usually comes from. Providers generally do not train on API traffic, and that is a real and meaningful protection. It is also answering a different question. Training is about retention. Residency is about transmission. Data can leave your jurisdiction and be deleted an hour later, and the transfer still happened.
Look at what you are about to send
The fastest way to settle this argument in a room is to print the payload before it goes anywhere.
Most agent code assembles messages in a function like this, and nobody ever looks at the output.
def build_messages(question: str) -> list[dict]:
docs = retrieve(question) # your database, your customers
return [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": question},
{"role": "tool", "content": "\n\n".join(d.text for d in docs)},
]
So put one line in front of the send.
msgs = build_messages("what is the balance on Aisha Rahman's account")
print(json.dumps(msgs, indent=2)) # <- this is the request body
And read what comes out.
[
{"role": "system", "content": "You are a support assistant for..."},
{"role": "user", "content": "what is the balance on Aisha Rahman's account"},
{"role": "tool", "content": "ACCT 4471-9920 | Aisha Rahman | balance AED 82,410.55 |
last txn 2026-07-21 | KYC tier 2 | ..."}
]
That third message is a customer record. It was not typed by anyone. Retrieval put it there, correctly, because that is the feature. And the whole array is about to become an HTTPS request to whichever region your provider resolves to.
Nobody decided to export that. It arrived as a consequence of a retrieval step someone added to improve answer quality.
Routing is the control
Once you can see the payload, the fix is ordinary engineering rather than policy.
def route(msgs: list[dict]) -> str:
"""Pick the model by what the payload contains, not by what is fastest."""
if carries_regulated_data(msgs): # your classifier, your definitions
return ONSHORE_MODEL # runs in infrastructure you control
return HOSTED_MODEL # everything else, the fast path
Two things worth saying about that function.
carries_regulated_data is the hard part and it is yours to define. It is also the part a policy
document cannot do for you, because a policy cannot inspect a message array at runtime.
And notice the default. The fast path is the fallback, which means an unclassified payload goes offshore. If that is the wrong default for you, invert it, and accept that unclassified traffic gets the slower model until someone classifies it. That choice is the actual decision, and it should be made deliberately rather than inherited from whichever branch you wrote first.
Why this lands hardest in the GCC
I build compliance tooling for GCC regulators, so this is the version I see most.
Regulated firms here operate under rules about where certain categories of data may be processed and under what conditions they may move across a border. The specific requirements differ by regulator and by data category, and I am not going to pretend a blog post can tell you which apply to you. What I will say is that the question is not exotic. It is one of the oldest questions in financial services technology, and it long predates AI.
What is new is how easy it now is to answer it wrongly by accident. Nobody signs off on "export customer records to a foreign jurisdiction on every keystroke." People sign off on "add an assistant that can answer questions about accounts," which is the same thing wearing better clothes.
When I model this in the platform I build, residency is not prose. It is a tag on the obligation itself, so a rule that requires processing to stay onshore can be checked against a vendor's declared deployment rather than argued about. That is the level of concreteness this needs: a fact you can evaluate, not a paragraph you can interpret.
Treat everything I have said about which obligations apply as scope, and as an advisory estimate. Verify with counsel before you rely on it.
The choice is not open versus closed
The sane pattern is a hybrid: validate the design with the most capable model you can reach, then move the parts that can move.
Applied to residency, that becomes a routing decision rather than a purchasing one.
Classify before you route. Not every prompt carries regulated data. A question about your public documentation and a question about a named customer's transactions are different payloads that happen to share an interface. If your architecture cannot tell them apart, it will treat both as the riskier one, or worse, as the safer one.
Put the boundary in the code path, not in the policy. This is the same argument as the system prompt one. "Staff have been told not to paste customer data" is a policy. A router that sends classified payloads to a model running in your own infrastructure is a control.
Know what your provider actually offers. Regional deployments, zero-retention endpoints and contractual commitments are all real and worth using. They are also all different from each other, and some of them are commitments about retention rather than location. Read which one you have.
Keep the option open. This is where swappability stops being an engineering convenience and becomes a compliance one. If your agent is welded to one provider, your residency posture is whatever that provider decides next quarter.
What this actually is
For the frameworks, this is not really an attack. It is exposure created by architecture, sitting on the MAESTRO Data Operations layer, and it is closest to the data-governance and cross-border-transfer obligation area rather than to any adversary technique.
Which is precisely why it gets missed. There is no incident, no alert and no attacker. There is a design decision, made for latency or capability, that quietly answered a regulatory question nobody realised was being asked.
The most expensive finding in an assessment is rarely the clever one. It is the one where the system works exactly as designed, and the design was the problem.