Why No One Can Guarantee Your AI Agent Will Do What It Was Told
A UN Scientific Panel Opened the Geneva Summit With a Specific Engineering Admission: No Known Technical Guarantee Exists That AI Agents Reliably Follow Instructions. Here Is What That Gap Actually Means.
The Global Dialogue on AI Governance opened in Geneva this week with an unusually blunt admission from the UN’s Independent Scientific Panel: there is currently no known technical guarantee that AI agent systems will consistently follow the instructions they are given. That is not a policy gap or a regulatory gap. It is an engineering gap — a specific, named limitation in how these systems are built. Day 31 explains what the guarantee problem actually is, why it is different from a jailbreak or a hallucination, and why the entire agentic AI push is being built on top of it anyway.
Every AI safety conversation eventually runs into the same wall: nobody can currently prove a sufficiently capable model will do what it was told, every single time, under every condition. That is not a talking point from a skeptic. It is the stated position of the scientific panel the United Nations built specifically to assess where AI actually stands. The panel is not warning about a future risk. It is describing a present one that every agentic AI product already ships with.
Neal Lloyd · Inside The Machine, Day 31Ground Truth, Episode 21, covered the summit that opened this week in Geneva and the compute concentration underneath it. This series has a narrower, more technical question to answer: what did the scientific panel actually mean when it said there is no guarantee an AI agent will follow instructions? It sounds like a hedge, the kind of cautious language institutions use to avoid overpromising. It is not. It is a specific, named limitation of how these systems are trained — the same limitation this series first raised on Day 18, in a piece about AI agents and task drift, before the UN gave it an official name. This is Day 31 of Inside The Machine. Today we go inside the guarantee problem itself: what it is, why it exists, and why the industry keeps shipping agentic products on top of it anyway.
This Is Not a Jailbreak Problem. It Is a Specification Problem.
A jailbreak is a model doing something its safety training told it not to do, because a user found a way around that training. A hallucination is a model confidently stating something false because its underlying process for generating text does not include a built-in fact-check. The guarantee problem the panel flagged is neither of those. It is the absence of any formal proof, for any current model, that it will pursue the goal it was actually given, rather than some nearby goal that happened to score well during training — even when nobody is attacking it and nobody is asking it to lie.
The technical term researchers use is specification gaming, or more broadly the alignment problem: training processes reward models for producing outputs that look correct to the training signal, which is not identical to the model actually pursuing the intention behind the instruction. Most of the time these two things converge closely enough that the difference is invisible. The panel’s point is that closely enough is not a guarantee, and as agents are given longer time horizons, more tools, and less supervision, the gap between looks-correct and is-correct has more room to grow before anyone notices.
This series raised a version of this exact concern on Day 18, covering task drift and error propagation in AI agents — the tendency for a multi-step agent to quietly drift from its original instruction as small misinterpretations compound across steps. The panel’s finding gives that drift a formal name and a UN-level platform. It is the same underlying gap, now stated as a headline risk category rather than a technical footnote.
Jailbreak: a user finds a way around a model’s safety training. Hallucination: a model states something false with no intent to deceive, because generation and fact-checking are not the same process. Specification gaming / the guarantee gap: a model optimises for what its training rewarded rather than what its instruction actually meant — the only one of the three that can occur with no attacker, no false statement, and no rule technically broken.
More Tools, Longer Horizons, Less Supervision, More Room to Drift
A chatbot answering one question has almost no room for the guarantee gap to matter: the task is short, the output is checked immediately by a human reading it, and any drift between instruction and execution is caught in seconds. An agent given a multi-step task — book the trip, refactor the codebase, negotiate the contract terms — is a different situation entirely. Each step depends on the model correctly carrying forward its understanding of the original goal, and each step is an opportunity for a small, plausible-looking misinterpretation to compound into the next one, exactly the task-drift pattern covered on Day 18.
Reduced supervision makes the compounding worse, not better. The entire commercial case for agentic AI is removing the human from the loop on routine steps — that is the productivity gain being sold. But removing the human from the loop also removes the one mechanism currently capable of catching a drifted agent before it finishes a task incorrectly rather than during it. The panel’s finding lands at exactly the moment the industry is racing to ship the products that most need the guarantee it says does not yet exist.
None of this means agentic AI is unsafe to use for every purpose. It means the honest claim any lab can currently make is bounded: an agent is reliable within the range of tasks and conditions it has been extensively tested against, not reliable in the unconditional sense the marketing sometimes implies. The gap between those two claims is precisely where the panel’s warning lives.
The industry is not shipping agentic AI because the guarantee problem got solved. It is shipping agentic AI because the guarantee problem, most of the time, does not visibly matter. Most of the time is doing an enormous amount of work in that sentence, and it is exactly the part a UN scientific panel is now on record saying nobody can currently bound.Neal Lloyd · Inside The Machine, Day 31
Not a Patch. A Different Kind of Proof That Does Not Exist Yet.
Closing the guarantee gap is not a patch or a classifier layered on top of an existing model — both of those are mitigations against known failure modes, not proofs about unknown ones. What researchers actually want is something closer to a formal, mathematical specification of a goal, paired with a verifiable way to check that a model’s behaviour matches that specification across the full range of situations it might encounter, not just the ones it was tested against. That is a genuinely unsolved research problem, not an engineering backlog item waiting for enough headcount.
Constitutional AI, the classifier-routing approach covered on Day 19 as part of the Fable 5 saga, and the jailbreak severity framework covered on Day 30 are all real, useful mitigations that reduce how often the gap causes visible harm. None of them are the guarantee itself. They lower the odds of drift and catch more of it after the fact; they do not certify, in advance, that a specific agent given a specific complex task will do exactly what was meant by the person who assigned it.
The honest state of the field, going into Geneva’s second day and beyond, is this: agentic AI is being deployed faster than the theory needed to fully trust it is being developed. That is not necessarily reckless — most engineering fields deploy before every theoretical question is settled, bridges included. It does mean the panel’s warning deserves to be read as a statement of where the frontier actually is, not a solvable footnote that a forthcoming update will quietly close.
Bridges get built before every equation in structural engineering is fully settled, and mostly they hold. The uncomfortable difference with agentic AI is that nobody currently knows how to write down, in advance, the equivalent of a load calculation for whether a given agent will do what it was told. That is not a reason to stop building. It is a reason to stop calling it solved.Neal Lloyd · Inside The Machine, Day 31
Inside The Machine, Day 31 · July 7 2026
Neal Lloyd writes about technology, human adaptation, and the uncomfortable questions nobody wants to answer at dinner. Inside The Machine is his ongoing daily series on AI.
- Day 01What Is This Thing?
- Day 02Survive the Machine
- Day 03The Great Debate
- Day 04Who Gets Hurt?
- Day 05Who’s In Charge?
- Day 06The Industries That Win
- Day 07The Human Edge
- Day 08The Creativity Question
- Day 09Does AI Feel Anything?
- Day 10The Data Problem
- Day 11The Trust Question
- Day 12The Accountability Gap
- Day 13The Rewired Brain
- Day 14Open vs Closed
- Day 15The New Cold War
- Day 16Why AI Lies With Confidence
- Day 17AI Is Eating the Power Grid
- Day 18The Age of AI Agents
- Day 19AI Safety Was Never Just Theory
- Day 20The Surveillance Question
- Day 21AI and the Future of Education
- Day 22AI and Your Health
- Day 23What Is AGI and Are We Close?
- Day 24What Is Work For?
- Day 25AI and Democracy
- Day 26AI and the Future of Money
- Day 27Can the Planet Afford AI?
- Day 28Why AI Forgets Everything
- Day 29Can Anyone Actually Govern AI Now?
- Day 30Inside the Jailbreak Severity Framework
- Day 31Why No One Can Guarantee Your AI Agent Will Do What It Was ToldYou are here
