Show HN: FrontierHarness Eval – 9 harness, same model, cost per pass varies 17x
A technical evaluation of nine different AI harnesses tested on identical software engineering tasks. The data highlights significant variations in cost-per-pass and performance efficiency.
Why it matters
This provides critical benchmarking data for developers and companies looking to optimize AI agent costs and reliability in software development workflows.
Pass rate 50.0% 55.6% 61.1% 66.7% $1 $2 $5 $10 $20 Codex 66.7% · $3.47 Codex 66.7% · $3.47 · 6m 43s DSH Creator 63.3% · $3.28 DSH Creator 63.3% · $3.28 · 6m 44s Claude Code 63.3% · $18.34 Claude Code 63.3% · $18.34 · 9m 38s Pi 60.0% · $2.43 Pi 60.0% · $2.43 · 7m 33s DSH PTC 60.0% · $4.58 DSH PTC 60.0% · $4.58 · 7m 44s DSH Standard 60.0% · $3.46 DSH Standard 60.0% · $3.46 · 6m 17s Oh My Pi 56.7% · $4.75 Oh My Pi 56.7% · $4.75 · 6m 46s Kimi Code 56.7% · $3.65 Kimi Code 56.7% · $3.65 · 7m 56s DSH Minimal 56.7% · $4.72 DSH Minimal 56.7% · $4.72 · 5m 41s Exo Harness 53.3% · $1.05 Exo Harness 53.3% · $1.05 · 6m 17s OpenCode 50.0% · $3.24 OpenCode 50.0% · $3.24 · 6m 27s…
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in