“People are consistently surprised that a model capable of extraordinary reasoning has, architecturally, less continuity of self than a goldfish. The goldfish at least has the same goldfish tomorrow. Every time you open a new conversation with most AI systems, you are talking to a model with no memory that you exist, no record of what you discussed yesterday, and no persistent sense of who it has been talking to across time — unless someone has specifically engineered a memory system to bolt onto that gap. The intelligence and the memory are not the same thing. Conflating them is the single most common misunderstanding people have about how these systems work.”
Neal Lloyd · Inside The Machine, Day 28Ask a frontier AI model to solve a complex multi-step mathematics problem, draft a legal brief, or debug a thousand-line codebase, and it will frequently perform at a level that exceeds what most human experts could produce in the same time. Close that conversation, open a new one, and ask the same model what you were just discussing, and it will have no idea. Not because it chose to forget, not because of a privacy setting, but because, in the most literal architectural sense, the conversation you just had no longer exists anywhere the model can access. This combination — extraordinary momentary capability paired with a near-total absence of persistent memory by default — is one of the most consistently confusing aspects of how large language models actually work, and it is the subject of more user frustration, more product design complexity, and more genuine architectural innovation than almost any other dimension of the technology. This is Day 28 of Inside The Machine. Today we explain, precisely, why AI forgets, what is actually happening when it appears to remember, and why this distinction matters enormously for how you should use these tools.
The Model Has No Memory. It Has a Window. The Difference Is Everything.
A large language model, at the moment it generates a response, is not consulting a memory in any sense a human would recognise. It is processing a single, finite block of text — called the context window — that contains everything the model currently has access to: the system instructions that define its behaviour, the conversation history up to that point, any documents or files that have been provided, and the specific prompt it is responding to. The model generates its response by processing this entire block simultaneously, identifying statistical patterns and relationships across all of it, and producing the most probable continuation. When the conversation ends and a new one begins, that context window is, by default, gone. The model has not learned anything from the previous conversation. It has not updated its underlying parameters. It has simply lost access to the text it was previously processing, because that text is no longer part of what is being fed into it.
This is architecturally different from how human memory works, and the difference is not merely philosophical — it has direct practical consequences. Human memory is, broadly, persistent and selective: experiences are encoded, consolidated, and retained (imperfectly, with distortion, but retained) across time, and can be retrieved later through cues, association, and effort. A language model’s “memory” within a single conversation is not memory in this sense at all. It is simply continued access to the same block of text. The model is not recalling what you said ten messages ago; it is re-reading it, because it is still sitting there in the context window, every single time the model generates a new response. This is computationally expensive — processing a longer context window requires more compute for every single response — which is why context windows have historically been limited, and why the size of a model’s context window has become one of the most closely watched specifications in the industry.
The scale of context windows has expanded dramatically. Early GPT-3 era models worked with a few thousand tokens of context — roughly the length of a long essay. Current frontier models, as covered in Day 13’s discussion of the cognitive divergence between human and AI attention spans, now operate with context windows reaching one to two million tokens — hundreds of thousands of words, enough to hold an entire codebase, a lengthy legal contract, or a multi-hour conversation transcript simultaneously. But the fundamental architectural fact remains unchanged regardless of size: it is still a window, not a memory. Close the window, and unless something has specifically been engineered to persist information beyond it, everything inside is gone.
2022 (GPT-3 era): ~4,000 tokens, approximately 3,000 words. 2023 (GPT-4 launch): ~8,000 to 32,000 tokens depending on variant. 2024: Context windows reach 128,000 to 200,000 tokens across major labs. 2025: Million-token context windows become available in frontier models (Gemini 1.5, Claude variants). 2026: Claude Opus 4.8 ships with a 1 million token default context window. GPT-5.6 Sol and OpenAI’s tier structure push further. The growth: roughly 250x in four years. What has not changed in that time: closing the conversation and opening a new one still, by default, discards everything, regardless of how large the window was while the conversation was active.
When an AI Assistant Seems to Remember You, Here Is What Is Actually Happening
The memory features that products like ChatGPT, Claude, and Gemini have introduced over the past two years — covered in detail in Episode 02’s examination of OpenAI’s Dreaming V3 system — are not a change to the underlying architecture described above. They are an engineered workaround, bolted onto a model that still, fundamentally, has no persistent memory of its own. Here is what is actually happening when an AI assistant appears to remember something about you across separate conversations.
The retrieval architecture. Memory features work by maintaining a separate, persistent database — entirely outside the language model itself — that stores specific facts, preferences, and summaries extracted from past conversations. When you start a new conversation, before the model generates any response, a separate process queries this database, retrieves whatever is judged relevant to the current conversation, and silently inserts it into the context window as part of the system instructions you do not see. The model is not remembering you. It is being handed a cheat sheet, freshly assembled, at the start of every conversation, summarising things you said in the past. The illusion of continuity is produced by this retrieval step, not by any persistent state inside the model itself.
The summarisation and extraction process. Populating that database requires a separate process — often another AI model call entirely — that reads through conversations after they end, identifies information judged worth retaining (your name, your job, a stated preference, a recurring project), and writes a condensed summary into the persistent store. This is the process that, as documented in Episode 02’s coverage of the CHI 2026 research, operates largely unilaterally: the system decides what is worth remembering about you, often without you explicitly instructing it to remember anything, and the summarisation process is itself a form of AI-generated content that can be incomplete, subtly wrong, or shaped by training biases about what kinds of information matter.
What this architecture cannot do. Because memory is retrieved and inserted rather than continuously present, it operates under the same context window constraints as everything else. A memory database with years of accumulated facts about you cannot simply be dumped wholesale into every conversation — it would consume the entire context window before the model could process your actual question. Retrieval systems therefore have to make judgement calls about what subset of stored memory is relevant to insert for any given conversation, and those judgement calls are themselves a source of error: the system might retrieve an irrelevant old fact while failing to surface something genuinely pertinent, producing exactly the kind of confidently wrong personalisation that feels uncanny rather than helpful.
When ChatGPT remembers your dog’s name from a conversation three months ago, the model itself has not retained anything. A separate retrieval system read a summary of that old conversation, judged it relevant to your new one, and silently inserted it into the model’s context window before it generated a response. The model is, in the truest architectural sense, meeting you for the first time, every single time — just with an increasingly detailed briefing document prepared for it in advance.Neal Lloyd · Inside The Machine, Day 28
The Consequences of Window-Based Memory for How You Should Actually Use AI
Context window position affects reliability. Research on long-context model performance has consistently identified what is sometimes called the “lost in the middle” effect: models reliably attend to information at the very beginning and very end of a context window, with measurably degraded recall and reasoning quality for information buried in the middle of a long document or conversation. This means that as a context window fills up over the course of a long conversation, the model’s effective grasp of everything that has been discussed is not uniform — details mentioned early in a long session can become functionally inaccessible even though they are technically still present in the window. For any task where consistency across a long working session matters, this is a genuine and underappreciated limitation.
Starting fresh is sometimes the better strategy, not a failure. Because long context windows degrade in reliability and consume more compute per response, there are many practical situations where deliberately starting a new conversation — rather than continuing an increasingly long one — produces better results, even though it means losing the accumulated context. A long debugging session that has wandered through several false starts and dead ends often produces better output if you summarise the actual problem cleanly and start fresh, rather than asking the model to continue reasoning inside a context window cluttered with the record of every wrong turn.
Memory features create a privacy surface that the architecture makes easy to overlook. Because the retrieval-and-insertion process happens silently, before you see any response, it is easy to forget that a persistent database of inferred facts about you exists at all, growing with every conversation, reviewable only through a settings page that most users never visit. The architectural invisibility of the memory system is, in part, what made the findings in Episode 02 so significant: the mechanism that makes AI assistants feel more genuinely helpful over time is the same mechanism quietly building a behavioural profile that operates largely outside conscious user awareness.
Understanding the architecture is the basis for using these tools well. The single most practically useful mental model for working with current AI systems is this: treat every new conversation as starting with a model that has access only to what is explicitly placed in front of it — the system instructions, any retrieved memory, the conversation so far, and your current message — and nothing else. If something matters and is not visibly present in the conversation, restate it. If a long conversation has become unwieldy, consider whether a fresh start with a clean, well-summarised prompt would outperform continuing inside a degraded context window. The model’s intelligence is real and substantial. Its memory, absent specific engineering to create the illusion of one, is not.
The architectural truth underneath every AI memory feature is this: there is no memory inside the model. There is only ever a window, freshly assembled, every single time. Understanding that distinction does not diminish what these systems can do. It clarifies exactly what they are doing — and explains why a system capable of reasoning at PhD level can, in the same conversation, behave as though it has never met you before, because in the most literal sense, it has not.Neal Lloyd · Inside The Machine, Day 28
Inside The Machine, Day 28 · July 1 2026
Neal Lloyd writes about technology, human adaptation, and the uncomfortable questions nobody wants to answer at dinner. Inside The Machine is his ongoing daily series on AI.
- Day 01What Is This Thing?
- Day 02Survive the Machine
- Day 03The Great Debate
- Day 04Who Gets Hurt?
- Day 05Who’s In Charge?
- Day 06The Industries That Win
- Day 07The Human Edge
- Day 08The Creativity Question
- Day 09Does AI Feel Anything?
- Day 10The Data Problem
- Day 11The Trust Question
- Day 12The Accountability Gap
- Day 13The Rewired Brain
- Day 14Open vs Closed
- Day 15The New Cold War
- Day 16Why AI Lies With Confidence
- Day 17AI Is Eating the Power Grid
- Day 18The Age of AI Agents
- Day 19AI Safety Was Never Just Theory
- Day 20The Surveillance Question
- Day 21AI and the Future of Education
- Day 22AI and Your Health
- Day 23What Is AGI and Are We Close?
- Day 24What Is Work For?
- Day 25AI and Democracy
- Day 26AI and the Future of Money
- Day 27Can the Planet Afford AI?
- Day 28Why AI Forgets EverythingYou are here
