PolybrainBench.

A living benchmark for how AI models disagree.

Most benchmarks measure whether a single AI can answer a question correctly. PolybrainBench measures something different: when you ask the same question to nine different AIs, where do they disagree, and how big is the gap?

Every verification cycle dispatches one claim to 9independently trained models in parallel. Every response is captured with its full text, its timing, and a cryptographic provenance stamp. No model sees any other model’s output during the cycle. No reviewer grades the cycle while it’s in flight. The pattern of who takes which position IS the data the benchmark publishes.

Every N cycles the daemon regenerates a paper from the current ledger and validates it against a separate six-model reviewer fleet. Two of those six are external anchors from provider families absent from the generator (Anthropic and Google), so their scores are fully independent. The other four are drawn from the generator to preserve the character of its strictest and most generous voices. Eleven unique models across seven independent training lineages touch the pipeline end to end. The paper regenerates and republishes itself.

The paper is free to read, cite, and build on. 10,452 verification cycles in the published dataset, growing every day. Hosted on Zenodo with a permanent identifier; the full ledger is on Hugging Face.

When AIs disagree, that’s where uncertainty actually lives.

The same claim. Nine models. In parallel. No answer key.

Growing by about two thousand cycles a day.

At paper v16 the corpus is a snapshot of a living artifact, not a fixed release. The measurement is the full response text from every model on every claim, not a summary or an accuracy score. The corpus is designed to grow: every 6 hours a scheduled daemon adds about five hundred new cycles, targeted at claim shapes where models are most likely to disagree.

Eleven models. Seven training lineages. Five API providers.

Two separate fleets do two separate jobs. The generator fleet of nine models dispatches against every claim in the corpus. The reviewer fleet of six models validates the paper about that corpus. Four reviewers are drawn from the generator to preserve its strictest and most generous voices; they grade the paper’s aggregate analysis of the full ledger rather than their own isolated per-claim responses. Two reviewers are external anchors — claude-sonnet-4-5 from Anthropic and gemini-2.5-pro from Google — from provider families absent from the generator, so they have no corpus contribution at all and their scores are fully independent of every training lineage that produced the data. Together: eleven unique models across seven independent training lineages, running on five live API billing relationships. Replicating the fleet from scratch means accounts, keys, quota, and monitoring with all five.

Per-model Q and A shown: generators reflect the historical v8 self-reviewed reading (preserved for comparison). External reviewers reflect the first Sprint 7 disjoint reading (v13). Composite = round(0.6 × mean(Q) + 0.4 × mean(A)) = 72 on vv16.

Three operations, applied recursively.

Cite it, download it, browse it.

@dataset{salvo_polybrainbench_2026,
  author    = {Salvo, Andy},
  title     = {PolybrainBench v8: A Living Benchmark for
               Cross-Model Consensus Verification of
               Natural-Language Claims},
  year      = 2026,
  publisher = {Zenodo},
  version   = {v8},
  doi       = {10.5281/zenodo.19546460},
  url       = {https://doi.org/10.5281/zenodo.19546460}
}

We also run a live verification API.

The same methodology, exposed as a pay-per-call endpoint for AI agents on the x402 economy. Free trust-profile reads. $0.01 per verification paid automatically via x402 on Base. Backed by the live PolybrainBench fleet.

const trust = await fetch(
  "https://trust.polylogicai.com/profile/0x..."
);
const { score } = await trust.json();
if (score < 50) skip(service);

const verified = await fetch(
  "https://trust.polylogicai.com/verify",
  {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ claim: aiOutput })
  }
);