The Vibe Security Index
AI app builders ship working software in an afternoon. This is our measurement of what the security posture of that code actually looks like, taken from public repositories generated by each builder and scanned with the same engine we run for customers.
Generated Oct 3, 2026 · 214 rules · 4 builders · 178 repositories scanned
Aggregate only. We never name a repository, an owner, or a URL, and we never publish a line of anyone's code. See the full methodology.
Download the round data (JSON). Vibe Security Index data is licensed under CC BY 4.0. Credit XploitScan and link to this page.
Application code — critical-finding rate
The share of scanned repositories with at least one critical finding in code that is not vendored scaffold. Lower is better. One critical finding is enough to cap a project's grade, so this is the number we lead with.
| # | Builder | Repos scanned | ≥1 critical | ≥1 high | Median findings |
|---|---|---|---|---|---|
| 1 | Base44 marker: @base44/sdk in package.json1,812 code-search hits for the marker · scaffold attribution | 50 10 excluded | 40% 20 of 50 | 98% 49 of 50 | 20.0 mean 88.4 |
| 2 | v0 marker: v0-user-next config merge in the Next.js config2,836 code-search hits for the marker · scaffold attribution | 50 3 excluded | 50% 25 of 50 | 82% 41 of 50 | 24.5 mean 77.8 |
| 3 | Bolt marker: bolt-vite-react-ts template in .bolt/config.json3,120 code-search hits for the marker · scaffold attribution | 28 166 excluded | 54% 15 of 28 | 100% 28 of 28 | 22.5 mean 173.2 |
| 4 | Lovable marker: lovable-tagger in package.json76,288 code-search hits for the marker · scaffold attribution | 50 11 excluded | 68% 34 of 50 | 100% 50 of 50 | 29.5 mean 119.3 |
Bar length is relative to the highest rate in this round, not to 100% — the printed percentage is the number that matters. Rates are computed over the repositories actually scanned, not over the matched universe.
Before you quote this table
- This measures public repositories carrying each builder's scaffold marker — not the builders' own products, and not the apps people never pushed to GitHub.
- Attribution is scaffold-level. The marker proves the project started from that builder; humans edit these repos afterwards, so some of what is measured is hand-written.
- Static analysis has false positives. A critical finding is something a human should look at, not a proven breach — and at these sample sizes a few points between builders is noise.
Grade distribution
Every scanned repository is graded with the same function a customer's scan uses, computed from application-code findings only. A single critical finding caps a project at D.
Base44
50 reposv0
50 reposBolt
28 reposLovable
50 reposWhat repeats
The findings that show up across the most distinct repositories per builder. These are the patterns worth fixing in a prompt, a template, or a review checklist — they are not one-off mistakes, they are the shape of what the tool generates.
Sorted by reach, not by severity — which is the wrong order for deciding what to fix, because the highest-volume checks here are code quality rather than security. Each builder has a page that reads its own list properly: what repeats, why that workflow produces it, and the prompt change that prevents it. Lovable, v0, Bolt, Base44.
Base44
- VC10647 repos / 50Magic Numbers in Code516 occurrences total
- VC02741 repos / 50Missing Security Meta Tags / Headers115 occurrences total
- VC05640 repos / 50Clickjacking — Missing X-Frame-Options65 occurrences total
- VC02040 repos / 50Missing Content Security Policy (CSP)56 occurrences total
- VC09728 repos / 50Console.log Left in Production Code99 occurrences total
- VC01619 repos / 50Unvalidated Redirect48 occurrences total
- VC00317 repos / 50API Route Missing Authentication101 occurrences total
- VC04316 repos / 50Timing-Unsafe Secret Comparison51 occurrences total
- VC01015 repos / 50Client-Side Only Authorization26 occurrences total
- VC03714 repos / 50Stack Traces Exposed in API Responses149 occurrences total
v0
- VC10640 repos / 50Magic Numbers in Code483 occurrences total
- VC09734 repos / 50Console.log Left in Production Code301 occurrences total
- VC00326 repos / 50API Route Missing Authentication532 occurrences total
- VC06523 repos / 50Unprotected Next.js API Route267 occurrences total
- VC15420 repos / 50API Route Without Request Body Validation162 occurrences total
- ENTROPY18 repos / 50High-Entropy String Detected (Possible Secret)357 occurrences total
- VC00814 repos / 50API Endpoint Without Rate Limiting15 occurrences total
- VC09813 repos / 50Synchronous File Operations78 occurrences total
- VC00713 repos / 50Potential Cross-Site Scripting (XSS)66 occurrences total
- VC02512 repos / 50Unsanitized Filename in File Operations52 occurrences total
Bolt
- VC02728 repos / 28Missing Security Meta Tags / Headers296 occurrences total
- VC05628 repos / 28Clickjacking — Missing X-Frame-Options156 occurrences total
- VC02028 repos / 28Missing Content Security Policy (CSP)148 occurrences total
- VC10618 repos / 28Magic Numbers in Code1,064 occurrences total
- VC09718 repos / 28Console.log Left in Production Code825 occurrences total
- VC03710 repos / 28Stack Traces Exposed in API Responses365 occurrences total
- VC1479 repos / 28Secret Logged to Console159 occurrences total
- VC1189 repos / 28Personally Identifiable Information in Logs157 occurrences total
- VC0439 repos / 28Timing-Unsafe Secret Comparison136 occurrences total
- VC0039 repos / 28API Route Missing Authentication132 occurrences total
Lovable
- VC02750 repos / 50Missing Security Meta Tags / Headers215 occurrences total
- VC05650 repos / 50Clickjacking — Missing X-Frame-Options118 occurrences total
- VC02050 repos / 50Missing Content Security Policy (CSP)107 occurrences total
- VC10649 repos / 50Magic Numbers in Code912 occurrences total
- VC09736 repos / 50Console.log Left in Production Code878 occurrences total
- VC00126 repos / 50Hardcoded API Key or Secret166 occurrences total
- VC08420 repos / 50Missing Subresource Integrity (SRI)23 occurrences total
- VC03718 repos / 50Stack Traces Exposed in API Responses708 occurrences total
- VC04617 repos / 50Session Fixation Risk27 occurrences total
- ENTROPY16 repos / 50High-Entropy String Detected (Possible Secret)109 occurrences total
Vendored scaffold — counted, published, not attributed
Many of these projects ship files nobody on the project wrote — chiefly UI component primitives copied in from a public library. (Build output and minified files are skipped by the scanner entirely.) Scanning that code and charging it to the builder would be the easiest way to make these numbers look dramatic, and it would be dishonest — the same files appear byte-identical across unrelated projects, so they measure the component library, not the builder.
So the split is mechanical and declared up front: findings in **/components/ui/**, **/ui/shadcn/**, **/shadcn/ui/**, **/shadcn/**, **/*.md, **/*.mdx, .claude/**, .agent/**, .agents/**, .cursor/**, .superpowers/**, .windsurf/**, **/.claude/**, **/.agent/**, **/.agents/**, **/.cursor/**, **/.superpowers/**, **/.windsurf/**, **/vendor/**, **/vendored/**, **/third_party/**, **/generated/**, **/__generated__/** are separated before any headline number is computed. They never touch a builder's grade, its critical rate, or its ranking. They are published here because hiding them would be its own kind of lie.
A large share of vendored findings are false positives on our side — a component library injecting a style block built from a module-level constant is CSS-variable theming, not an injection sink, and a rule that flags it is our bug, not the builder's. Treating that as evidence about anyone's security posture would be indefensible.
| Builder | % with a vendored critical | Median vendored findings | Most common vendored rule |
|---|---|---|---|
| Base44 | 32% | 1.0 | ENTROPY High-Entropy String Detected (Possible Secret) (16 repos) |
| v0 | 78% | 5.0 | ENTROPY High-Entropy String Detected (Possible Secret) (41 repos) |
| Bolt | 4% | 0.0 | VC097 Console.log Left in Production Code (4 repos) |
| Lovable | 88% | 5.0 | ENTROPY High-Entropy String Detected (Possible Secret) (45 repos) |
These figures are excluded from every other number on this page. They are here so you can check the size of what we removed rather than take our word for it.
Do not read this column as a security ranking. It measures how much of a component library each builder copies in. A builder that scaffolds the full shadcn/ui suite inherits the two files that generate nearly all of these findings; one that ships a handful of hand-rolled primitives inherits none of them. A 0% rate here means the builder vendors less code, not that it writes safer code — and since most of what is in this column is our own false positive, a high rate is closer to a statement about our rules than about anyone's security.
The sample
How many public repositories carry each builder's marker, how many we sampled, and how confident we are that the marker means what we think it means.
| Builder | Universe | Sampled | Scanned | Excluded | Median files | Attribution |
|---|---|---|---|---|---|---|
| Base44 | 1,812 | 50 | 50 | 10 | 90.0 | scaffold attribution |
| v0 | 2,836 | 50 | 50 | 3 | 93.0 | scaffold attribution |
| Bolt | 3,120 | 28 | 28 | 166 | 57.0 | scaffold attribution |
| Lovable | 76,288 | 50 | 50 | 11 | 113.5 | scaffold attribution |
Universe is GitHub code search's total_count for the builder's marker at sampling time — a count of matching files (including backup copies such as package.json.bak), not of distinct repositories, and an estimate rather than a census.
Median files matters more than it looks. A smaller project has fewer places for a critical finding to occur, so a builder whose users ship smaller apps scores better on every rate above without necessarily generating safer code. This round's medians run from 57 to 113.5 files, so the column is published rather than left for a reader to work out.
Movement
Round-over-round movement is not shown yet. The history this section is computed from currently holds only round 2026-07, and later rounds have not been added to it.
How this round was run
GitHub code search for each builder's scaffold marker, deduped to one row per repository, screened against the eligibility rules below, then the first N survivors in GitHub's own result order. No hand-picking: the sampler never sees scan results, and the scanner never sees the eligibility reasons. Markers by builder — Lovable: lovable-tagger in package.json; v0: v0-user-next config merge in the Next.js config; Bolt: bolt-vite-react-ts template in .bolt/config.json; Base44: @base44/sdk in package.json.
A repository is eligible when:
- Public, non-fork, non-archived, non-disabled repositories only.
- Repositories flagged as GitHub templates are excluded.
- Repositories owned by the builder's own organization are excluded (product and docs, not user apps).
- The scaffold marker must appear in one of the builder's live marker files (package.json for Lovable and Base44, the Next.js config for v0, .bolt/config.json for Bolt). A mention in a README, a lockfile or a backup copy of the manifest is not evidence the repo was scaffolded by the builder.
- Repositories whose name, description or topics identify them as a tutorial, course, boilerplate, starter kit, template, workshop or awesome-list are excluded.
- Size bounds: 40 KB to 250,000 KB. Below the floor there is no app; above the ceiling it is a monorepo or asset dump rather than one vibe-coded app.
- Must contain at least 5 non-vendored source files, measured from the git tree before cloning.
- Repositories that fail to clone or scan are counted as excluded, never as clean.
Vendored paths this round: **/components/ui/**, **/ui/shadcn/**, **/shadcn/ui/**, **/shadcn/**, **/*.md, **/*.mdx, .claude/**, .agent/**, .agents/**, .cursor/**, .superpowers/**, .windsurf/**, **/.claude/**, **/.agent/**, **/.agents/**, **/.cursor/**, **/.superpowers/**, **/.windsurf/**, **/vendor/**, **/vendored/**, **/third_party/**, **/generated/**, **/__generated__/**
Scanning is the deterministic rule set only — the full 214-rule paid set as of this round, no AI filter. The scanner has since grown to 223 rules; a free scan runs 30 of them. Its measured precision and recall are published on the benchmark page, including the cases it misses. No repository's code is ever executed, built, or installed.
The AI false-positive filter is deliberately off. It would produce lower, more flattering numbers — it exists precisely to drop findings a model judges uninteresting — but it is not deterministic, so a second run could return a different result and nobody outside this company could check the difference. An index that cannot be re-run is an assertion, not a measurement. The cost is a higher false-positive rate in these figures than a customer with the filter enabled would see, which is the trade we would rather make, and the reason the vendored split above matters so much. Full methodology →
What this does not tell you
- Attribution is scaffold-level, not authorship-level. The marker proves the project was created from the builder's scaffold; it does not prove every line was generated by it. Humans edit these repos afterwards, and some of what is measured is human-written.
- The sample is public GitHub only. Apps that were never pushed, were pushed privately, or were deployed straight from the builder without a repo are invisible, and there is no reason to assume they are similar.
- GitHub code search caps at 1,000 results per query, so for builders with a larger universe the sample is drawn from the first 1,000 results in GitHub's relevance order, not uniformly at random from the whole universe.
- The 'vendored' bucket is broader than the word suggests: it covers copied UI component libraries, markdown documentation and committed AI-agent tooling directories — everything the builder's code generation is not accountable for. Those findings are reported, not discarded.
- Static analysis has false positives. Findings are not confirmed exploits, and a critical finding is a thing a human should look at, not a proven breach.
- Project size is a confound and it is not small: median scanned files per repository ranges from 57 (Bolt) to 113.5 (Lovable). A smaller project has fewer places for a critical finding to occur, so a builder whose users ship smaller apps scores better on every rate here without necessarily generating safer code. The per-builder median is published in the sample table so this can be read directly rather than inferred.
- The vendored-critical column is a fact about scaffold composition, not about security. It largely reflects how much of a component library each builder copies in: a builder that ships the full shadcn/ui suite inherits the two files that generate nearly all vendored findings, while one that ships a few hand-rolled primitives inherits none. A 0% vendored rate means the builder vendors less code, not that it produces safer code.
- Some rules fire on scaffold files the builder ships unmodified (a missing Content-Security-Policy on the stock index.html, for instance) where the real control lives in host headers the scanner cannot see. Those inflate the high-severity counts for every builder roughly equally and do not affect the critical-severity headline.
- Per-repo rows are anonymized by design. The numbers are reproducible by re-running the pipeline; the individual repositories behind them are deliberately not recoverable, because publishing them would disclose live vulnerabilities in other people's projects.
- Files over 1.5 MB are skipped, and repositories are scanned up to 8,000 files. 2 of 178 repositories hit the file cap and were measured partially, which understates their finding counts.
- Round size: 178 repositories across 4 builders. Differences of a few percentage points between builders are inside the noise of a sample this size.
The three that most change how the leaderboard should be read are repeated directly beneath it, because the limitations are the part most likely to be dropped when a chart gets screenshotted. This is the complete list.
What we will not publish
This index reports aggregates and nothing else. We do not publish repository names, owners, URLs, code snippets, or any value that could identify a project — a public list of live vulnerabilities in other people's apps is a target list, not research. Nothing in a scanned repository is executed, installed, or built, and no secret value is ever copied out of one.
If you build one of these platforms and think a number here is wrong, we want to hear it before your users do. Email admin@xploitscan.com and we will re-run the round with your correction and publish the result either way. Our disclosure policy covers anything that needs to stay private.
Scan your own build
Aggregates are interesting. Your repository is the one that matters. The same engine that produced this page runs on your code in about a minute. A first scan needs no signup and runs the 30-rule free set; the full rule set behind this round comes with a paid plan.