Bench'd

Bench'd is a benchmark authority for AI memory systems. Its site describes it as a neutral benchmark and a scoreboard for this area, with public leaderboards and a published methodology intended to support reproducible, independently run evaluation.

The service runs memory systems through benchmark protocols that measure recall, temporal correctness, failure traces, and how efficiently past experience improves future performance. It says every run is cryptographically signed and publicly verifiable, and it adds spend attestation through ProofMeter, marked patent pending. The site also distinguishes between verified results and listed or self-reported claims, and notes that self-reported systems are labeled as not independently run by Bench'd. The benchmark coverage includes tracks such as Conversational Memory, Knowledge Brain, and Agent Memory, with examples shown for systems like LlamaIndex Memory, gbrain, and Letta.

Bench'd lists 64 systems indexed and 13 independently scored, and it says 46 systems are awaiting adapters. It presents benchmark index pages, latest signed receipts, scoring models, methodology documentation, and a trust system with defined tiers. The scoring model separates deterministic exact-match scoring for verified results from an LLM-judged nuance score for synthesis and open-ended recall. It also says that eight benchmark specs are published and 23 failure codes are documented.

The site is delivered as a web service with pages for the leaderboard, benchmarks, docs, and blog, and it invites users to claim a system, connect an official endpoint, and verify results against the public harness. It also offers newsletter updates for new benchmark results and methodology changes.