CadEval measures end-to-end CAD synthesis: given a textual prompt, a model must produce a CAD program (e.g., OpenSCAD-style) that renders to an object matching the specification. The benchmark reports both success rates and geometric fidelity via metrics like rendering success, Chamfer/Hausdorff distances, and volume/consistency checks.

The tasks stress long-horizon reasoning, tool-augmented generation, and an understanding of geometry and constraints. Strong results require not only producing syntactically valid code but also achieving shape similarity that satisfies strict quantitative thresholds.