Back to models

o1-preview

ECI#146
135
OpenAI
Sep. 12, 2024
Closed weights

o1-preview is a closed-weights model from OpenAI, released September 12, 2024. It has an ECI score of 135 and ranks 146th of 260 tracked models. At the time of its release, it ranked 2nd of 79 tracked models.

o1-preview vs select alternatives

DomainBenchmark
o1-preview
OpenAI
Sep. 12, 2024
Closed weights
GPT-4o (May 2024)
OpenAI
May 13, 2024
Closed weights
Claude 3.5 Sonnet
Anthropic
Jun. 20, 2024
Closed weights
Llama 3.1-405B
Meta AI
Jul. 23, 2024
Open weights
ECI scoreBest score 167
135129130129
Ranking#146#166#162#169
Conceptual reasoning
SimpleBenchBest score 82%
42%
n/a
28%
23%
Mathematics
MATH Level 5Best score 98%
82%
51%
52%
50%
OTIS Mock AIME 2024-2025Best score 100%
31%
6%
7%
10%
Science
GPQA DiamondBest score 96%
50%
49%
54%
51%
Software engineering
METR Time HorizonsBest score 17 hr 25 min
20 min 20 sec9 min 10 sec11 min 24 sec—
CybenchBest score 93%
10%
n/a
18%
8%
WeirdML (v2)Best score 94%
48%
n/a
31%
21%

Model metadata

General
OrganizationOpenAI
Release dateSep. 12, 2024
Model accessibilityClosed weights
CountryUnited States of America
Knowledge cutoff2023-09-30
Provider notesA new series of reasoning models for solving hard problems.
Pricing
Input price$15 / 1M tokens
Output price$60 / 1M tokens
Training
ParametersUnknown
Training computeUnknown
Training hardwareUnknown
Training timeUnknown
Training dataset sizeUnknown