The Vibe Security Index
AI app builders ship working software in an afternoon. This is the measurement nobody publishes: what the security posture of that code actually looks like, taken from public repositories generated by each builder and scanned with the same rule set we run for customers.
Generated Jul 27, 2026 · 212 rules · 4 builders · 87 repositories scanned
Aggregate only. We never name a repository, an owner, or a URL, and we never publish a line of anyone's code. See the full methodology.
Application code — critical-finding rate
The share of scanned repositories with at least one critical finding in code that is not vendored scaffold. Lower is better. One critical finding is enough to cap a project's grade, so this is the number that decides whether a shipped app is safe to point at real users.
| # | Builder | Repos scanned | ≥1 critical | ≥1 high | Median findings |
|---|---|---|---|---|---|
| 1 | Bolt marker: bolt-vite-react-ts template in .bolt/config.json1,312 public repos matched · scaffold attribution | 12 86 excluded | 42% 5 of 12 | 100% 12 of 12 | 13.0 mean 25.0 |
| 2 | v0 marker: v0-user-next config merge in the Next.js config1,372 public repos matched · scaffold attribution | 25 7 excluded | 56% 14 of 25 | 80% 20 of 25 | 36.0 mean 44.5 |
| 3 | Base44 marker: @base44/sdk in package.json1,248 public repos matched · scaffold attribution | 25 8 excluded | 60% 15 of 25 | 100% 25 of 25 | 29.0 mean 134.0 |
| 4 | Lovable marker: lovable-tagger in package.json10,284 public repos matched · scaffold attribution | 25 10 excluded | 76% 19 of 25 | 100% 25 of 25 | 51.0 mean 228.1 |
Bar length is relative to the highest rate in this round, not to 100% — the printed percentage is the number that matters. Rates are computed over the repositories actually scanned, not over the matched universe.
Before you quote this table
- This measures public repositories carrying each builder's scaffold marker — not the builders' own products, and not the apps people never pushed to GitHub.
- Attribution is scaffold-level. The marker proves the project started from that builder; humans edit these repos afterwards, so some of what is measured is hand-written.
- Static analysis has false positives. A critical finding is something a human should look at, not a proven breach — and at these sample sizes a few points between builders is noise.
Grade distribution
Every scanned repository gets the same letter grade a customer would see, computed from application-code findings only. A single critical finding caps a project at D.
Bolt
12 reposv0
25 reposBase44
25 reposLovable
25 reposWhat repeats
The findings that show up across the most distinct repositories per builder. These are the patterns worth fixing in a prompt, a template, or a review checklist — they are not one-off mistakes, they are the shape of what the tool generates.
Bolt
- VC02712 repos / 12Missing Security Meta Tags / Headers44 occurrences total
- VC05612 repos / 12Clickjacking — Missing X-Frame-Options23 occurrences total
- VC02012 repos / 12Missing Content Security Policy (CSP)22 occurrences total
- VC1067 repos / 12Magic Numbers in Code24 occurrences total
- VC0977 repos / 12Console.log Left in Production Code16 occurrences total
- VC0033 repos / 12API Route Missing Authentication45 occurrences total
- VC0373 repos / 12Stack Traces Exposed in API Responses8 occurrences total
- VC0652 repos / 12Unprotected Next.js API Route14 occurrences total
- VC1482 repos / 12Secret Leaked in Error Response7 occurrences total
- VC1542 repos / 12API Route Without Request Body Validation7 occurrences total
v0
- VC10622 repos / 25Magic Numbers in Code175 occurrences total
- VC09720 repos / 25Console.log Left in Production Code101 occurrences total
- VC00311 repos / 25API Route Missing Authentication55 occurrences total
- VC06511 repos / 25Unprotected Next.js API Route53 occurrences total
- VC15410 repos / 25API Route Without Request Body Validation36 occurrences total
- VC00810 repos / 25API Endpoint Without Rate Limiting13 occurrences total
- VC1008 repos / 25N+1 Query Pattern Detected23 occurrences total
- VC0077 repos / 25Potential Cross-Site Scripting (XSS)39 occurrences total
- VC0256 repos / 25Unsanitized Filename in File Operations16 occurrences total
- VC0436 repos / 25Timing-Unsafe Secret Comparison14 occurrences total
Base44
- VC02725 repos / 25Missing Security Meta Tags / Headers189 occurrences total
- VC05625 repos / 25Clickjacking — Missing X-Frame-Options97 occurrences total
- VC02025 repos / 25Missing Content Security Policy (CSP)95 occurrences total
- VC10624 repos / 25Magic Numbers in Code559 occurrences total
- VC01019 repos / 25Client-Side Only Authorization66 occurrences total
- VC01618 repos / 25Unvalidated Redirect128 occurrences total
- VC10014 repos / 25N+1 Query Pattern Detected284 occurrences total
- VC09714 repos / 25Console.log Left in Production Code205 occurrences total
- VC03713 repos / 25Stack Traces Exposed in API Responses468 occurrences total
- VC10412 repos / 25Empty Catch Block173 occurrences total
Lovable
- VC10625 repos / 25Magic Numbers in Code834 occurrences total
- VC02725 repos / 25Missing Security Meta Tags / Headers161 occurrences total
- VC05625 repos / 25Clickjacking — Missing X-Frame-Options90 occurrences total
- VC02025 repos / 25Missing Content Security Policy (CSP)80 occurrences total
- VC09721 repos / 25Console.log Left in Production Code824 occurrences total
- VC00117 repos / 25Hardcoded API Key or Secret154 occurrences total
- VC04615 repos / 25Session Fixation Risk24 occurrences total
- VC10013 repos / 25N+1 Query Pattern Detected226 occurrences total
- VC03712 repos / 25Stack Traces Exposed in API Responses572 occurrences total
- ENTROPY11 repos / 25High-Entropy String Detected (Possible Secret)83 occurrences total
Vendored scaffold — counted, published, not attributed
Every one of these projects ships dozens of files nobody on the project wrote: UI component primitives copied in from a public library, build output, minified bundles. Scanning that code and charging it to the builder would be the easiest way to make these numbers look dramatic, and it would be dishonest — the same files appear byte-identical across unrelated projects, so they measure the component library, not the builder.
So the split is mechanical and declared up front: findings in **/components/ui/**, **/ui/shadcn/**, **/shadcn/ui/**, **/*.md, **/*.mdx, .claude/**, .agent/**, .cursor/**, .superpowers/**, .windsurf/**, **/.claude/**, **/.agent/**, **/vendor/**, **/vendored/**, **/third_party/**, **/generated/**, **/__generated__/** are separated before any headline number is computed. They never touch a builder's grade, its critical rate, or its ranking. They are published here because hiding them would be its own kind of lie.
A large share of vendored findings are false positives on our side — a component library injecting a style block built from a module-level constant is CSS-variable theming, not an injection sink, and a rule that flags it is our bug, not the builder's. Treating that as evidence about anyone's security posture would be indefensible.
| Builder | % with a vendored critical | Median vendored findings | Most common vendored rule |
|---|---|---|---|
| Bolt | 0% | 0.0 | — |
| v0 | 80% | 5.0 | ENTROPY High-Entropy String Detected (Possible Secret) (22 repos) |
| Base44 | 84% | 5.0 | ENTROPY High-Entropy String Detected (Possible Secret) (21 repos) |
| Lovable | 88% | 5.0 | ENTROPY High-Entropy String Detected (Possible Secret) (22 repos) |
These figures are excluded from every other number on this page. They are here so you can check the size of what we removed rather than take our word for it.
The sample
How many public repositories carry each builder's marker, how many we sampled, and how confident we are that the marker means what we think it means.
| Builder | Universe | Sampled | Scanned | Excluded | Attribution |
|---|---|---|---|---|---|
| Bolt | 1,312 | 12 | 12 | 86 | scaffold attribution |
| v0 | 1,372 | 25 | 25 | 7 | scaffold attribution |
| Base44 | 1,248 | 25 | 25 | 8 | scaffold attribution |
| Lovable | 10,284 | 25 | 25 | 10 | scaffold attribution |
Universe is the count of public repositories matching the builder's marker at sampling time. Code-search result sets are capped, so treat it as a floor on the real population, not a census.
How this round was run
GitHub code search for each builder's scaffold marker, deduped to one row per repository, screened against the eligibility rules below, then the first N survivors in GitHub's own result order. No hand-picking: the sampler never sees scan results, and the scanner never sees the eligibility reasons. Markers by builder — Lovable: lovable-tagger in package.json; v0: v0-user-next config merge in the Next.js config; Bolt: bolt-vite-react-ts template in .bolt/config.json; Base44: @base44/sdk in package.json.
A repository is eligible when:
- Public, non-fork, non-archived, non-disabled repositories only.
- Repositories flagged as GitHub templates are excluded.
- Repositories owned by the builder's own organization are excluded (product and docs, not user apps).
- The scaffold marker must appear in one of the builder's live marker files (package.json for Lovable and Base44, the Next.js config for v0, .bolt/config.json for Bolt). A mention in a README, a lockfile or a backup copy of the manifest is not evidence the repo was scaffolded by the builder.
- Repositories whose name, description or topics identify them as a tutorial, course, boilerplate, starter kit, template, workshop or awesome-list are excluded.
- Size bounds: 40 KB to 250,000 KB. Below the floor there is no app; above the ceiling it is a monorepo or asset dump rather than one vibe-coded app.
- Must contain at least 5 non-vendored source files, measured from the git tree before cloning.
- Repositories that fail to clone or scan are counted as excluded, never as clean.
Vendored paths this round: **/components/ui/**, **/ui/shadcn/**, **/shadcn/ui/**, **/*.md, **/*.mdx, .claude/**, .agent/**, .cursor/**, .superpowers/**, .windsurf/**, **/.claude/**, **/.agent/**, **/vendor/**, **/vendored/**, **/third_party/**, **/generated/**, **/__generated__/**
Scanning is the deterministic rule set only — 212 rules, no AI filter, the same engine and version behind a scan you run yourself. Its measured precision and recall are published on the benchmark page, including the cases it misses. No repository's code is ever executed, built, or installed.
The AI false-positive filter is deliberately off. It would produce lower, more flattering numbers — it exists precisely to drop findings a model judges uninteresting — but it is not deterministic, so a second run could return a different result and nobody outside this company could check the difference. An index that cannot be re-run is an assertion, not a measurement. The cost is a higher false-positive rate in these figures than a customer with the filter enabled would see, which is the trade we would rather make, and the reason the vendored split above matters so much. Full methodology →
What this does not tell you
- Attribution is scaffold-level, not authorship-level. The marker proves the project was created from the builder's scaffold; it does not prove every line was generated by it. Humans edit these repos afterwards, and some of what is measured is human-written.
- The sample is public GitHub only. Apps that were never pushed, were pushed privately, or were deployed straight from the builder without a repo are invisible, and there is no reason to assume they are similar.
- GitHub code search caps at 1,000 results per query, so for builders with a larger universe the sample is drawn from the first 1,000 results in GitHub's relevance order, not uniformly at random from the whole universe.
- The 'vendored' bucket is broader than the word suggests: it covers copied UI component libraries, markdown documentation and committed AI-agent tooling directories — everything the builder's code generation is not accountable for. Those findings are reported, not discarded.
- Static analysis has false positives. Findings are not confirmed exploits, and a critical finding is a thing a human should look at, not a proven breach.
- Some rules fire on scaffold files the builder ships unmodified (a missing Content-Security-Policy on the stock index.html, for instance) where the real control lives in host headers the scanner cannot see. Those inflate the high-severity counts for every builder roughly equally and do not affect the critical-severity headline.
- Per-repo rows are anonymized by design. The numbers are reproducible by re-running the pipeline; the individual repositories behind them are deliberately not recoverable, because publishing them would disclose live vulnerabilities in other people's projects.
- Files over 1.5 MB are skipped, and repositories are scanned up to 8,000 files. No repository in this round hit the file cap, so every repository was measured whole.
- Round size: 87 repositories across 4 builders. Differences of a few percentage points between builders are inside the noise of a sample this size.
The three that most change how the leaderboard should be read are repeated directly beneath it, because the limitations are the part most likely to be dropped when a chart gets screenshotted. This is the complete list.
What we will not publish
This index reports aggregates and nothing else. We do not publish repository names, owners, URLs, code snippets, or any value that could identify a project — a public list of live vulnerabilities in other people's apps is a target list, not research. Nothing in a scanned repository is executed, installed, or built, and no secret value is ever copied out of one.
If you build one of these platforms and think a number here is wrong, we want to hear it before your users do. Email admin@xploitscan.com and we will re-run the round with your correction and publish the result either way. Our disclosure policy covers anything that needs to stay private.
Scan your own build
Aggregates are interesting. Your repository is the one that matters. The same rule set that produced this page runs on your code in about a minute — no signup required for a first scan.