Technology
A coding-agent benchmark tests whether JavaScript frameworks beat vibes
A benchmark compares coding agents building the same Pokémon application across JavaScript frameworks and meta-frameworks. It measures build time, token use, code size, conformance, Lighthouse performance and accessibility, using mostly five-run medians. The results suggest meaningful differences in cost and maintainability, while performance metrics favor some multi-page approaches and remain sensitive to Lighthouse variability. The author stresses that the small app, chosen agent, framework versions and limited iterations prevent broad framework rankings, but argues for evidence-based evaluation.
Editorial summary · Publisher article
What matters
- The benchmark tested repeated builds of the same application against a conformance suite.
- SvelteKit, Gracile and Angular were more expensive to build in the reported runs.
- Hono plus htmx achieved similar functionality to Angular with substantially less code in the author’s comparison.
- The author warns that Lighthouse and the chosen content-focused app favor some architectures.
What the benchmark can and cannot show
The experiment isolates one relatively simple application and one coding agent, so its results do not establish a universal best framework. Agent randomness, inconsistent Lighthouse measurements and uneven use of framework tools also affected the comparisons.
Why code size matters with agents
The author argues that smaller codebases can reduce the token and review burden of future agent-assisted changes. That makes maintainability a direct operating cost in agentic development, although the benchmark does not treat succinctness as the only engineering objective.
Context & caveats
- The findings are the author’s experiment, not an independent replication or a comprehensive evaluation of real-world applications.
Why this made the edition
The author presents a reproducible benchmark design, multiple iterations, explicit limitations and concrete measurements of coding-agent cost, quality and maintainability. Its conclusions are appropriately bounded to the tested app and setup.
Source material