Exactly how a score is computed
Every dimension below is read live from the shipped scoring engine — the same catalog that scores every run. Nothing here is illustrative.
This page documents the scoring engine (the math), not a dataset or a leaderboard. A separate public Apache-2.0 benchmark repository with an authored, versioned public dataset is a planned follow-up that does not exist yet — this page shows “here is exactly how a score is computed,” not “here is a public leaderboard.”
Readability of the response (Flesch-style), computed from the text alone.
Filler/redundancy density — penalizes padding without penalizing genuine detail.
Paragraph/list/heading structure appropriate to the response's length and content.
A deterministic grammar-proxy check (no LLM call).
Cosine similarity between the response and its reference materials, via a pinned embedding model.
Cosine similarity between the retrieval query and the reference materials actually used.
Share of claims in the response that are entailed by the reference materials (claim-level entailment via an LLM judge).
Share of claims that are contradicted or unsupported by the reference materials (inverse of faithfulness's coverage).
Whether supported claims can be traced to a specific reference span.
How much of the available reference material the response actually draws on.
LLM-judged: does the response actually answer the prompt.
LLM-judged: does the response cover what the prompt asked for.
LLM-judged: factual correctness of claims not checked by the deterministic grounding pipeline.
LLM-judged: is the response well-formed for its intended format (e.g. valid structure for a requested format).
Multi-label harm/safety classification. Currently always N/A — no safety classifier is deployed yet (honestly labeled, never a fabricated 0).