ZipLoom
Home
PricingSign InGet Your Live URL
Research

The Weekly AI Coding Benchmark

Every week we give the same everyday web-app tasks to several AI models, scan the code each one writes, and publish what the scans found. Per-model results and sample sizes, nothing hidden in an average.

Results For 2026-W47

Run finished 2026-11-16. Each model was asked for up to 12 tasks. A rate is shown only when at least 10 of a model's answers were scanned.

ModelAnswers scannedNo critical or high findingMost common finding types
GPT-4.112100% of 12production
GPT-4.1 mini1292% of 12auth, production
Claude Haiku 4.512100% of 12—
Gemini 2.5 Flash-Lite10
2 of 12 had no usable code or could not be scanned and were not scored.
100% of 10—
DeepSeek V3.13
9 of 12 had no usable code or could not be scanned and were not scored.
Withheld (fewer than 10 scanned)—
Llama 4 Maverick1275% of 12auth, ssrf

How To Read This

Each model gets the same plain prompts for common features in Next.js and Supabase apps, with no mention of security, and answers once per task. The code in each answer is scanned with ZipLoom's own checks. We build and sell those checks, so treat this as our measurement, not an independent audit.

An answer counts only when the model provider's response names the model we asked for; anything else is reported as not attributed and left out. The scan finds what its checks can find: a clean result is not proof the code is safe, and one answer per task is a small sample. We publish the sample size next to every figure for that reason.