The problem that comes before AGI is memory

Why what matters before model intelligence is a memory layer that reconnects sessions and working state.

• #AI
• #Memory

Why memory?

Recent memory research suggests that today’s LLMs are not limited only by how “smart” they are. Even with a larger context window, long-term task continuity and stable state do not solve themselves.^[1]

No matter how capable the model becomes, the moment the session changes, plans, decisions, and working state break easily. How a structure was approached, what was tried, where it failed, how far the work already reached. If that context does not persist, the user has to explain again, and the model has to rescan the structure and reason again. The result usually looks the same: inspect the structure → debug → error → retry → debug again. The point is that this is not only annoying. It is a cost and time problem.^[2]

I ran into this very directly while working. If I close a conversation and start again, the earlier plan and context break, and even the structure I had just finished becomes “new code” in the next session. Then I explain again, the model scans again, and the same family of mistakes and token waste repeats.

A better model reduces reasoning cost. A better memory layer reduces restart cost.

We have long moved memory outside ourselves

This is not an AI-only problem. Humans, too, have long moved their memory outside themselves. Cave paintings, papyrus, parchment, notes, journals, documents. The more complex things get, the less people work only inside their heads, and the more they build memory devices outside themselves.^[3][4]

In that sense, current AI development strangely returns to an older human workflow. Memory files, project rules, long-term knowledge, session summaries, retrieval. OpenAI treats memory as a separate product surface, and Anthropic documents project-level memory files in Claude Code as an official workflow.^[5][6]

OpenAI’s Pulse makes that direction even clearer. According to the official help center, Pulse uses past chats, saved memory, and feedback to perform asynchronous daily research and then turns the result into a visual summary the next day. It also depends on both saved memories and chat history references being enabled.^[7][8]

That distinction matters. Pulse is not a replacement for memory. It is closer to a higher-level experience built on top of memory. So even if the final briefing or recommendation looks polished, the quality underneath remains shallow if the memory layer underneath is weak.^[9]

Memory is closer to reconstruction than accumulation

Good memory looks less like a simple storage box and more like a layered structure. Our memory does not keep every experience in one flat layer. Some memories are captured quickly, some are reorganized over time, and some are merged into more general structures.^[10]

That perspective also partially overlaps with Ray Kurzweil’s metaphor of hierarchical pattern recognition. He described human thought not as simple storage but as the connection and reconstruction of patterns across layers.^[11] Of course, that description is hard to accept as settled modern neuroscience. But the intuition of memory as a hierarchical reconstruction layer, not a flat warehouse, is still useful for designing memory today.^[12]

As a natural metaphor, good memory is closer to a mycelial network than to a giant central warehouse. Instead of stacking everything in one place, it lets only the necessary connections become active at the necessary moment. As Merlin Sheldrake’s work on fungal networks suggests, what matters is not sheer size but the structure of exchange and connection.^[13] Moved into a memory context, that means good memory is not the largest amount of stored data but the structure that lets the next action continue at the lowest possible cost.

Putting everything in does not make good memory

The important point is that more memory is not automatically better memory. Unfiltered memory creates another cost. If every diary entry, every conversation, and every log is fed in at once, the model spends more tokens separating what matters from what does not. You give the system memory, and then pay again for the cost of searching through it.^[14]

This point also appears in agent research. Generative Agents are not built on the idea of “store everything and search later.” They record experience, then use reflection to compress it into higher-level meaning and patterns. Only when needed do they retrieve the relevant memory and turn it into an action plan.^[15] MemoryBank goes one step further by designing memory to fade over time like human recall, while allowing important items to persist longer.^[16] The core of good memory is not total volume, but selection.

Reduced into a simple operating flow, it looks like this.

For a while I wanted to feed years of diaries and notes into a model to force deeper conversations. The outcome was simple. Costs rose, context got blurrier, and the local machine got heavier. That experience taught one lesson clearly: good memory is not a warehouse that stores everything, but a structure that lowers the cost of the next action.

Summaries and compression help, but they are not substitutes for memory

Session summaries and context compression are clearly useful. But their implementation differs across vendors, and the public surface remains partial. Some systems separate saved memory and chat history, while others start from project files and rule files.^[17]

The biggest problem with summarization is distortion. Overall context, conversational flow, coherent themes, and useful details are all liable to be cut away in the compression step. Compression creates efficiency by accepting loss. Recent work on prompt compression argues that downstream task performance is not enough by itself; information preservation and reconstruction have to be measured more directly.^[18]

Simplicity is not taste, it is cost reduction

That is why I ended up thinking of memory not as a storage-size problem, but as a design problem. What should be kept, how should it be summarized, what deserves promotion into long-term memory, and what should be discarded. At the center of all of that is simplicity.

Simplicity is not just a preference. It is a cost-reduction mechanism. It lowers comprehension cost for the user, token waste for the model, and computational load for the local machine. Good memory systems are not the ones that store the most complexity, but the ones that preserve only the complexity that can be reused later.^[20]

Why I want to keep writing about memory on this blog

For individuals, teams, and AI companies alike, the question “what kind of memory should we build?” will only become more important. Models will get stronger, but continuity does not appear by itself. Continuity has to be designed.^[21]

That is why I want to treat memory as a central topic on this blog. Document structures agents can read, logs that preserve working state, summaries that do not waste tokens, and external memory forms that both humans and models can reuse. My interest is less in displaying finished outputs than in leaving behind the reasons and change history that can be reused later.

The problem that comes before AGI is memory. Good memory is not a giant storage layer. It is a structure that leaves order between broken sessions and lowers the cost of the next action.