Long-term memory for AI assistants
Remember everything.
Inject almost nothing.
A memory layer with the highest published score on LoCoMo, reading a fifth of the context of the systems next to it. Measured on three public benchmarks, every question, with every model and token count named.
LoCoMo · all 1,540 questions
Does memory survive a long conversation? Highest score we can find published, at 22% of Mem0's context tokens per question.
BEAM-1M · all 700 questions
Does memory hold over a million tokens of history? 4.4 points ahead at 40% of the context. Hindsight is ahead of both, at 23,900 tokens.
LongMemEval-S · all 500 questions
Does memory stay right when facts change? Level with Mem0 at 36% of the context. Every other score of 93 or more uses a frontier model to answer.
hanumemAI: answers by qwen3.8-flash with thinking. Judges: gpt-4o-mini with Mem0's open harness prompt on LoCoMo (91.6 under a stricter judge); qwen3.7-flash with the paper's prompts on LongMemEval (91.8 under gpt-4o-mini) and on BEAM. Other systems' figures are as their vendors published them, with their own readers and judges; none is reproduced here, and no two vendors share a judge. Read the tables as context, not a leaderboard.
Accuracy, against what it had to read
Any system can score well by handing the model everything. Up is more accurate; left is less context. Each point is one system that publishes both numbers.
What we claim, and what we do not
We claim
- The highest published LoCoMo score. 96.1 on all 1,540 questions; Agent Zero 93.6, EverMemOS 93.05, Mem0 92.5.
- The best score per context token on all three benchmarks. 1,500 to 3,000 tokens per question; systems near or above us read 6,700 to 46,000.
- Ahead of Mem0 on BEAM-1M. 68.5 against 64.1 on the full 700 questions.
- The top group on LongMemEval-S with a small open model. 91.8 to 94.0 depending on the judge.
We do not claim
- First place on LongMemEval-S. agentmemory reports 96.2; Agent Zero and Chronos 95.6.
- First place on BEAM. Hindsight reports 73.9 at one million tokens.
- A like-for-like win. No two vendors share a judge, and no number here, ours included, has been reproduced by a third party.
- A held-out result. The system was tuned on the question sets it is scored on.
What is public, and what is on request
Public
- Every score, with its reader, judge and context tokens
- How it was earned: 162 logged runs, 23 changes kept
- The results paper, in HTML and PDF
- The subject, taught from zero in the open chapters of the book
On request
- The library
- The full technical paper
- The run log, and what each experiment changed
- A test on your own conversations
Go deeper
Request access
Tell me what you are building.
hanumemAI is not open source. If you want to use it, test it on your own conversations, or read the full paper, write to me here. I read every message myself.
- No account needed
- Your message goes to Madhu and nowhere else