> ## Documentation Index > Fetch the complete documentation index at: https://supermemory-vorflux-slack-workspace-override-consent.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # MemoryBench > Open-source framework for benchmarking memory providers — including your own Benchmarking memory systems is hard, and most comparisons you'll find aren't apples-to-apples — different datasets, different judges, different prompts, cherry-picked runs. **MemoryBench** ([`supermemoryai/memorybench`](https://github.com/supermemoryai/memorybench), MIT licensed) is the open-source framework we built to fix that: the same benchmark questions, the same pipeline, and the same judges run against every provider, so a comparison actually means something. We open-sourced it so you don't have to take our word for anything — you can run it yourself, against your own memory implementation, on the datasets that match your use case. Clone it, run it against your own data, or read the source for exactly how each provider is scored. *** ## Benchmark your own memory system MemoryBench ships a **Claude Code skill** that automates the entire process of benchmarking a custom memory implementation — yours — against Supermemory, Mem0, and Zep. Point it at your code and it handles discovery, integration, and the run: 1. Asks a few questions about your setup (provider name, where your memory code lives, which benchmark, which competitors, how many questions) 2. Analyzes your memory code to find its init, ingest, and search methods 3. Generates a provider adapter and registers it with the framework 4. Runs the full benchmark against your chosen competitors 5. Reports accuracy, latency, and context-token results side by side ```bash theme={null} # From your project root /memorybench ``` No manual TypeScript required to get a first result — see [Adding a Provider](/memorybench/extend-provider) for what the skill generates and how to do it by hand. *** ## How it works Every run goes through the same checkpointed pipeline, regardless of provider or benchmark: ``` INGEST → SEARCH → ANSWER → EVALUATE → REPORT ``` Ingestion can take hours for large datasets, and API calls fail — so every phase checkpoints independently and a run resumes from the last completed step instead of starting over. Chunk-based semantic search LLM-powered memory extraction Knowledge graph construction | Benchmark | Tests | Source | | --------------- | ------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | | **LoCoMo** | Fact recall across extended, multi-session conversations — single-hop, multi-hop, temporal, adversarial | [snap-research/locomo](https://github.com/snap-research/locomo) | | **LongMemEval** | Long-term memory across sessions, including knowledge that gets updated mid-conversation | [xiaowu0162/longmemeval](https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned) | | **ConvoMem** | Personalization, preference learning, and reference resolution within a conversation | [Salesforce/ConvoMem](https://huggingface.co/datasets/Salesforce/ConvoMem) | None of these fit your use case? [Build your own benchmark](/memorybench/extend-benchmark) — providers, benchmarks, and judges are all pluggable. Judging is judge-agnostic too — score a run with GPT-4o, Claude, Gemini, or any model you configure, so results aren't an artifact of one evaluator's bias. *** ## Read next Test on scenarios that actually match your product, not just the built-in datasets. Register your own memory system so it can be scored and compared. MemScore, judge scoring, and how to read a report. Found a bug, or want a provider/benchmark we don't support yet?