Overview

Epoch’s Benchmark Reviews are independent reviews we conduct of external benchmarks. This documentation describes the rubric we use to evaluate each benchmark and answers FAQs. Each benchmark will be labeled with one of the following verdicts:

• Verified Verified: The benchmark can broadly be interpreted as described. Any errors that exist do not substantially affect the results.
• Flawed Flawed: The benchmark has one or more substantive flaws that we believe users need to be aware of to accurately interpret results.
• Not enough info Not enough information: There is not enough information for us to review the benchmark.
• By Epoch By Epoch: We will not review benchmarks we created due to conflict of interest.

If you have questions or would like to suggest additional benchmarks, contact us at reviews@epoch.ai.