You ask ChatGPT to draft a quote for a client, then move on to the next job. Later that afternoon, you come back expecting it to remember the details of your conversation. But it doesn’t. The context is gone. You have to start again.
This isn’t a glitch. It’s a fundamental limitation of the AI tools many Irish businesses are starting to rely on. And a new research paper from leading AI labs has put a number on just how serious the problem really is.
The Memory Problem That Nobody Talks About
Researchers from top universities recently introduced something called PM-Bench — a test designed to measure ‘prospective memory’ in AI systems. That’s a fancy term for the ability to remember to do something at the right time, in the right context, while handling other tasks. Think of it like this: you’re wiring a house and you ask an assistant to remind you to order a specific fitting when you reach a certain point in the job. A good assistant remembers. A great one remembers without being reminded again.
So how did the latest AI models perform? Even the best system — built on GPT-5.4 — scored only 65.1% on this test. That means in more than a third of cases, the AI simply forgot to follow through on what it was asked to do.
What This Means for an Irish Small Business
This isn’t just an academic curiosity. If you’re using AI tools to help manage your business — drafting emails, organising customer follow-ups, generating quotes, or tracking project tasks — the memory gap is something you need to work around, not ignore.
Think about the typical day in a trades business. You get a call about a leaky roof in Galway. You ask your AI assistant to draft a response. Then a supplier calls about delayed materials. Then you need to check pricing for a job in Athlone. Each time you switch context, the AI’s ‘memory’ of your previous task fades — sometimes entirely.
For an Irish business owner, that means: if you ask an AI to do five things, there’s a decent chance it will forget at least one of them, especially if you’ve been doing other work in between.
Why Does AI Struggle With Memory?
Most AI models today work within a fixed ‘context window’ — a short-term memory that gets overwritten as conversations continue. When you start a new chat or jump between topics, the model doesn’t carry information forward the way a human assistant would.
The PM-Bench test simulated a full week of tasks, requiring AI agents to track delayed intentions while handling ongoing activities. It’s similar to how a busy tradie or landlord juggles multiple jobs, tenants, and suppliers across a working week. And across every configuration tested, the AI systems struggled.
The researchers found that no single strategy for improving memory worked across all models. Some performed better with specific prompting techniques, others with extended context windows. But none came close to reliable human-level prospective memory.
Practical Steps to Work Around AI’s Memory Limits
Does this mean you should stop using AI for your business? Absolutely not. But it does mean you should work with its strengths and around its weaknesses. Here’s how:
One task at a time. Instead of giving the AI a long list of instructions, break requests into separate conversations. A fresh chat for each distinct task.
Write it down. If you need the AI to remember something important, put it in a shared document or note that you paste into each new conversation. Tools like Notion, Obsidian, or even a simple text file work well for this.
Use task-specific tools. Rather than asking a general-purpose chatbot to manage your calendar, quotes, and customer follow-ups, use dedicated business tools that happen to have AI features built in. They’re designed to keep track of individual tasks properly.
Check before you trust. Treat AI outputs as a first draft, not a final answer. Especially if the task involved multiple steps or information given earlier in the conversation.
The paper was recently published on arXiv under the title “PM-Bench: Evaluating Prospective Memory in LLM Agents.” It’s a useful reminder that while AI is getting smarter every month, it still can’t match the simple reliability of a good notebook and a clear head.
For now, think of AI as a very clever junior assistant — great at many things, but not quite ready to be trusted with remembering everything.