SciCode is a benchmark of research-level coding problems curated by scientists across physics, mathematics, chemistry, biology, and materials science. Each problem is broken into subproblems whose solutions must be implemented in Python and pass the benchmark’s test cases, so it measures whether models can turn scientific knowledge into working code. The scores shown here are measured independently by Artificial Analysis.

We source results from the Artificial Analysis SciCode leaderboard, which reports a subproblem accuracy score: the share of subproblems whose generated code passes the test cases. Our chart shows that score, where higher is better.

Artificial Analysis runs SciCode itself rather than republishing the original authors’ numbers, and reports a single accuracy metric per model configuration. We include every model for which Artificial Analysis publishes a directly measured (non-estimated) SciCode score, and keep the provider and Artificial Analysis model identifier in the data export for auditability.