Detection Benchmark
We publish the raw numbers on how well our scanner performs against a labeled corpus of vulnerable and clean code samples. Checked daily and rerun when rules or fixtures change (at least weekly); updates go live with the next deploy.
Last run: Oct 13, 2026 · Corpus size: 615 fixtures (329 vulnerable, 286 clean)
The numbers above are for all 223 rules (paid plans). On the free plan's 30 rules, the same corpus scores precision no forbidden-rule hits and recall 21.3% (TP 127 / FP 0 / FN 470). Most fixtures target vulnerabilities only the paid rules detect.
Trend over time
90 days · since Jun 13, 2026Recall from the last benchmark run of each day. The corpus grows over time, so a flat or rising line means detection kept pace as new fixtures were added.
Precision is not charted. Under this corpus definition it has no forbidden-rule hits.
Corpus: 232 → 615 fixtures. Deterministic scanner only (excludes the optional AI false-positive filter).
How we score
Every fixture in packages/cli/test-fixtures/ is labeled with expectedFindings (rules that must fire, with file and line range) and mustNotFire (rules that must not fire anywhere in the fixture). The runner scans each fixture and counts:
- True positive (TP) — a finding whose rule, file, and line fall inside an expected entry.
- False negative (FN) — an expected entry with no matching finding.
- False positive (FP) — a finding for a rule explicitly listed as mustNotFire on a clean fixture.
Findings for rules that are neither expected nor listed as mustNotFire aren't counted as FPs, on vulnerable and clean fixtures alike — they're treated as adjacent observations, neither penalized nor rewarded. That is why this definition can look perfect; the head-to-head section below gives the figure when every finding on a clean fixture counts. Precision and recall are micro-averaged across the whole corpus. The runner lives at scripts/benchmark.js in the repository. Checked daily and rerun when rules or fixtures change (at least weekly); updates go live with the next deploy.
About this baseline. The corpus is deliberately small and curated — growing it is ongoing work. A perfect score on a small corpus is not a claim that the scanner catches everything; it's the floor below which we will not regress. We will keep adding harder cases, and over time the score will become a more demanding indicator.
Held-out set — 8/8 blind and 16/16 after fixes
The corpus above is ours — we wrote the rules and the fixtures, so a high score partly measures how well we test ourselves. This set is different: 16 real vulnerabilities pulled from public, intentionally-vulnerable projects (OWASP NodeGoat, Juice Shop, DVNA) and one public prototype-pollution example that none of our rule authors wrote, with the projects' hint comments stripped. We expect a lower number here — that's the point, and 8 of 8 (a small sample) is the honest one. When the first 16 cases were each first scored, before any rule was changed for them, XploitScan caught 8 of 16. The 8 misses were then studied and the rules changed to catch them, so those 8 are excluded from the blind figure above — a case you have studied cannot measure whether a rule generalises to code you have not. They still appear in the table below. Why we built this →
On the free plan's 30 rules alone, the same cases score 7 of 16 (5 of 8 blind).
The same corpus is scored against other scanners with the same class-level methodology: did the tool flag this class of vulnerability anywhere in the fixture? XploitScan 100.0% · Semgrep 50.0% · Bearer 56.3% (all three scored over the full 16-case corpus. XploitScan's rules were changed after we studied 8 of these cases; Semgrep and Bearer were run unmodified. Before that tuning, XploitScan caught 8 of 16 (50.0%).)
| Vulnerability class | Source | XploitScan | Semgrep | Bearer |
|---|---|---|---|---|
| Command injection | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | missed | ✓ caught |
| Insecure deserialization | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | ✓ caught | missed |
| Code injection (eval) | OWASP/NodeGoat — app/routes/contributions.js (Apache-2.0) | ✓ caught | ✓ caught | ✓ caught |
| IDOR / broken access control (BOLA) | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | missed | missed |
| Hardcoded credentials | juice-shop/juice-shop — lib/insecurity.ts (MIT) | ✓ caught | ✓ caught | ✓ caught |
| Open redirect | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | ✓ caught | ✓ caught |
| Path traversal | juice-shop/juice-shop — routes/fileServer.ts (MIT) | ✓ caught | ✓ caught | ✓ caught |
| Prototype pollution | kimmobrunfeldt/lodash-merge-pollution-example — index.js (ISC) | ✓ caught | missed | missed |
| SQL injection | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | ✓ caught | ✓ caught |
| SQL injection | juice-shop/juice-shop — routes/login.ts (MIT) | ✓ caught | ✓ caught | ✓ caught |
| SSRF | OWASP/NodeGoat — app/routes/research.js (Apache-2.0) | ✓ caught | missed | missed |
| Server-side template injection | juice-shop/juice-shop — routes/dataErasure.ts (MIT) | ✓ caught | missed | missed |
| Weak password hashing | juice-shop/juice-shop — lib/insecurity.ts, routes/login.ts (MIT) | ✓ caught | missed | ✓ caught |
| NoSQL injection | OWASP/NodeGoat — app/data/allocations-dao.js (Apache-2.0) | ✓ caught | missed | missed |
| Cross-site scripting (DOM) | juice-shop/juice-shop — frontend search-result.component.ts (MIT) | ✓ caught | missed | missed |
| XXE | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | ✓ caught | ✓ caught |
XploitScan detects every case in this set; the misses shown are Semgrep's and Bearer's. Corpus + sources in the public repo under test-fixtures/held-out/.
Per-rule scores
| Rule | TP | FP | FN | Recall | F1 |
|---|---|---|---|---|---|
| ENTROPY | 7 | 0 | 0 | 100.0% | — |
| VC001 | 13 | 0 | 0 | 100.0% | — |
| VC002 | 1 | 0 | 0 | 100.0% | — |
| VC003 | 17 | 0 | 0 | 100.0% | — |
| VC004 | 1 | 0 | 0 | 100.0% | — |
| VC005 | 3 | 0 | 1 | 75.0% | 85.7% |
| VC006 | 9 | 0 | 0 | 100.0% | — |
| VC007 | 5 | 0 | 0 | 100.0% | — |
| VC008 | 2 | 0 | 0 | 100.0% | — |
| VC009 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC010 | 2 | 0 | 0 | 100.0% | — |
| VC011 | 21 | 0 | 0 | 100.0% | — |
| VC012 | 1 | 0 | 0 | 100.0% | — |
| VC013 | 2 | 0 | 0 | 100.0% | — |
| VC014 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC015 | 1 | 0 | 0 | 100.0% | — |
| VC016 | 1 | 0 | 0 | 100.0% | — |
| VC017 | 2 | 0 | 0 | 100.0% | — |
| VC018 | 1 | 0 | 0 | 100.0% | — |
| VC019 | 1 | 0 | 0 | 100.0% | — |
| VC020 | 1 | 0 | 0 | 100.0% | — |
| VC021 | 1 | 0 | 0 | 100.0% | — |
| VC022 | 1 | 0 | 0 | 100.0% | — |
| VC023 | 4 | 0 | 0 | 100.0% | — |
| VC024 | 1 | 0 | 0 | 100.0% | — |
| VC025 | 4 | 0 | 0 | 100.0% | — |
| VC026 | 1 | 0 | 0 | 100.0% | — |
| VC027 | 1 | 0 | 0 | 100.0% | — |
| VC028 | 1 | 0 | 0 | 100.0% | — |
| VC029 | 3 | 0 | 0 | 100.0% | — |
| VC030 | 1 | 0 | 0 | 100.0% | — |
| VC031 | 3 | 0 | 0 | 100.0% | — |
| VC032 | 5 | 0 | 0 | 100.0% | — |
| VC033 | 1 | 0 | 0 | 100.0% | — |
| VC034 | 2 | 0 | 0 | 100.0% | — |
| VC035 | 2 | 0 | 0 | 100.0% | — |
| VC036 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC037 | 4 | 0 | 0 | 100.0% | — |
| VC038 | 2 | 0 | 0 | 100.0% | — |
| VC039 | 2 | 0 | 0 | 100.0% | — |
| VC040 | 1 | 0 | 0 | 100.0% | — |
| VC041 | 6 | 0 | 0 | 100.0% | — |
| VC042 | 10 | 0 | 0 | 100.0% | — |
| VC043 | 1 | 0 | 0 | 100.0% | — |
| VC044 | 6 | 0 | 1 | 85.7% | 92.3% |
| VC045 | 1 | 0 | 0 | 100.0% | — |
| VC046 | 2 | 0 | 0 | 100.0% | — |
| VC047 | 2 | 0 | 0 | 100.0% | — |
| VC048 | 3 | 0 | 0 | 100.0% | — |
| VC049 | 9 | 0 | 0 | 100.0% | — |
| VC050 | 1 | 0 | 0 | 100.0% | — |
| VC051 | 2 | 0 | 0 | 100.0% | — |
| VC052 | 1 | 0 | 0 | 100.0% | — |
| VC053 | 3 | 0 | 0 | 100.0% | — |
| VC054 | 2 | 0 | 0 | 100.0% | — |
| VC055 | 1 | 0 | 0 | 100.0% | — |
| VC056 | 1 | 0 | 0 | 100.0% | — |
| VC057 | 1 | 0 | 0 | 100.0% | — |
| VC058 | 1 | 0 | 0 | 100.0% | — |
| VC059 | 1 | 0 | 0 | 100.0% | — |
| VC060 | 2 | 0 | 0 | 100.0% | — |
| VC061 | 3 | 0 | 0 | 100.0% | — |
| VC062 | 1 | 0 | 0 | 100.0% | — |
| VC063 | 6 | 0 | 0 | 100.0% | — |
| VC064 | 4 | 0 | 0 | 100.0% | — |
| VC065 | 2 | 0 | 0 | 100.0% | — |
| VC066 | 1 | 0 | 0 | 100.0% | — |
| VC067 | 2 | 0 | 0 | 100.0% | — |
| VC068 | 3 | 0 | 0 | 100.0% | — |
| VC069 | 2 | 0 | 0 | 100.0% | — |
| VC070 | 1 | 0 | 0 | 100.0% | — |
| VC071 | 1 | 0 | 0 | 100.0% | — |
| VC072 | 1 | 0 | 0 | 100.0% | — |
| VC073 | 1 | 0 | 0 | 100.0% | — |
| VC074 | 1 | 0 | 0 | 100.0% | — |
| VC075 | 1 | 0 | 0 | 100.0% | — |
| VC076 | 3 | 0 | 0 | 100.0% | — |
| VC077 | 1 | 0 | 0 | 100.0% | — |
| VC078 | 1 | 0 | 0 | 100.0% | — |
| VC079 | 2 | 0 | 0 | 100.0% | — |
| VC080 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC081 | 2 | 0 | 0 | 100.0% | — |
| VC082 | 5 | 0 | 0 | 100.0% | — |
| VC083 | 1 | 0 | 0 | 100.0% | — |
| VC084 | 2 | 0 | 0 | 100.0% | — |
| VC085 | 2 | 0 | 0 | 100.0% | — |
| VC086 | 2 | 0 | 0 | 100.0% | — |
| VC087 | 1 | 0 | 0 | 100.0% | — |
| VC088 | 2 | 0 | 0 | 100.0% | — |
| VC089 | 2 | 0 | 0 | 100.0% | — |
| VC090 | 1 | 0 | 0 | 100.0% | — |
| VC091 | 1 | 0 | 0 | 100.0% | — |
| VC092 | 2 | 0 | 0 | 100.0% | — |
| VC093 | 2 | 0 | 0 | 100.0% | — |
| VC094 | 10 | 0 | 0 | 100.0% | — |
| VC095 | 4 | 0 | 0 | 100.0% | — |
| VC096 | 5 | 0 | 0 | 100.0% | — |
| VC097 | 1 | 0 | 0 | 100.0% | — |
| VC098 | 2 | 0 | 0 | 100.0% | — |
| VC099 | 8 | 0 | 0 | 100.0% | — |
| VC100 | 1 | 0 | 0 | 100.0% | — |
| VC101 | 4 | 0 | 0 | 100.0% | — |
| VC102 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC103 | 4 | 0 | 0 | 100.0% | — |
| VC104 | 4 | 0 | 0 | 100.0% | — |
| VC105 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC106 | 3 | 0 | 1 | 75.0% | 85.7% |
| VC107 | 1 | 0 | 0 | 100.0% | — |
| VC108 | 1 | 0 | 0 | 100.0% | — |
| VC109 | 1 | 0 | 0 | 100.0% | — |
| VC110 | 1 | 0 | 0 | 100.0% | — |
| VC111 | 1 | 0 | 0 | 100.0% | — |
| VC112 | 2 | 0 | 0 | 100.0% | — |
| VC113 | 1 | 0 | 0 | 100.0% | — |
| VC114 | 1 | 0 | 0 | 100.0% | — |
| VC115 | 2 | 0 | 0 | 100.0% | — |
| VC116 | 1 | 0 | 0 | 100.0% | — |
| VC117 | 2 | 0 | 0 | 100.0% | — |
| VC118 | 7 | 0 | 0 | 100.0% | — |
| VC119 | 1 | 0 | 0 | 100.0% | — |
| VC120 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC121 | 4 | 0 | 0 | 100.0% | — |
| VC122 | 3 | 0 | 0 | 100.0% | — |
| VC123 | 11 | 0 | 0 | 100.0% | — |
| VC124 | 12 | 0 | 0 | 100.0% | — |
| VC125 | 8 | 0 | 0 | 100.0% | — |
| VC126 | 1 | 0 | 0 | 100.0% | — |
| VC127 | 8 | 0 | 0 | 100.0% | — |
| VC128 | 2 | 0 | 0 | 100.0% | — |
| VC129 | 8 | 0 | 0 | 100.0% | — |
| VC130 | 3 | 0 | 1 | 75.0% | 85.7% |
| VC131 | 3 | 0 | 0 | 100.0% | — |
| VC132 | 1 | 0 | 0 | 100.0% | — |
| VC133 | 2 | 0 | 0 | 100.0% | — |
| VC134 | 2 | 0 | 0 | 100.0% | — |
| VC135 | 1 | 0 | 0 | 100.0% | — |
| VC136 | 1 | 0 | 0 | 100.0% | — |
| VC137 | 1 | 0 | 0 | 100.0% | — |
| VC138 | 1 | 0 | 0 | 100.0% | — |
| VC139 | 3 | 0 | 0 | 100.0% | — |
| VC140 | 2 | 0 | 0 | 100.0% | — |
| VC141 | 2 | 0 | 0 | 100.0% | — |
| VC142 | 1 | 0 | 0 | 100.0% | — |
| VC143 | 2 | 0 | 0 | 100.0% | — |
| VC144 | 2 | 0 | 0 | 100.0% | — |
| VC145 | 1 | 0 | 0 | 100.0% | — |
| VC146 | 1 | 0 | 0 | 100.0% | — |
| VC147 | 7 | 0 | 0 | 100.0% | — |
| VC148 | 1 | 0 | 0 | 100.0% | — |
| VC149 | 1 | 0 | 0 | 100.0% | — |
| VC150 | 2 | 0 | 0 | 100.0% | — |
| VC151 | 2 | 0 | 0 | 100.0% | — |
| VC152 | 1 | 0 | 0 | 100.0% | — |
| VC153 | 1 | 0 | 0 | 100.0% | — |
| VC154 | 1 | 0 | 0 | 100.0% | — |
| VC155 | 4 | 0 | 0 | 100.0% | — |
| VC156 | 1 | 0 | 0 | 100.0% | — |
| VC157 | 2 | 0 | 0 | 100.0% | — |
| VC158 | 3 | 0 | 0 | 100.0% | — |
| VC159 | 2 | 0 | 0 | 100.0% | — |
| VC160 | 1 | 0 | 0 | 100.0% | — |
| VC161 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC162 | 2 | 0 | 0 | 100.0% | — |
| VC163 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC164 | 1 | 0 | 0 | 100.0% | — |
| VC165 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC166 | 1 | 0 | 0 | 100.0% | — |
| VC167 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC168 | 1 | 0 | 0 | 100.0% | — |
| VC169 | 2 | 0 | 0 | 100.0% | — |
| VC170 | 1 | 0 | 0 | 100.0% | — |
| VC171 | 1 | 0 | 0 | 100.0% | — |
| VC172 | 1 | 0 | 0 | 100.0% | — |
| VC173 | 1 | 0 | 0 | 100.0% | — |
| VC174 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC175 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC176 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC177 | 2 | 0 | 0 | 100.0% | — |
| VC178 | 1 | 0 | 0 | 100.0% | — |
| VC179 | 2 | 0 | 0 | 100.0% | — |
| VC180 | 2 | 0 | 0 | 100.0% | — |
| VC181 | 3 | 0 | 0 | 100.0% | — |
| VC182 | 2 | 0 | 0 | 100.0% | — |
| VC183 | 3 | 0 | 0 | 100.0% | — |
| VC184 | 1 | 0 | 0 | 100.0% | — |
| VC185 | 1 | 0 | 0 | 100.0% | — |
| VC186 | 2 | 0 | 0 | 100.0% | — |
| VC187 | 6 | 0 | 0 | 100.0% | — |
| VC188 | 3 | 0 | 0 | 100.0% | — |
| VC189 | 1 | 0 | 0 | 100.0% | — |
| VC190 | 1 | 0 | 0 | 100.0% | — |
| VC191 | 1 | 0 | 0 | 100.0% | — |
| VC192 | 1 | 0 | 0 | 100.0% | — |
| VC193 | 2 | 0 | 0 | 100.0% | — |
| VC194 | 1 | 0 | 0 | 100.0% | — |
| VC195 | 2 | 0 | 0 | 100.0% | — |
| VC196 | 3 | 0 | 0 | 100.0% | — |
| VC197 | 3 | 0 | 0 | 100.0% | — |
| VC198 | 1 | 0 | 0 | 100.0% | — |
| VC199 | 1 | 0 | 0 | 100.0% | — |
| VC200 | 1 | 0 | 0 | 100.0% | — |
| VC201 | 2 | 0 | 0 | 100.0% | — |
| VC202 | 9 | 0 | 0 | 100.0% | — |
| VC203 | 2 | 0 | 0 | 100.0% | — |
| VC204 | 1 | 0 | 0 | 100.0% | — |
| VC205 | 1 | 0 | 0 | 100.0% | — |
| VC206 | 1 | 0 | 0 | 100.0% | — |
| VC207 | 1 | 0 | 0 | 100.0% | — |
| VC208 | 1 | 0 | 0 | 100.0% | — |
| VC209 | 1 | 0 | 0 | 100.0% | — |
| VC210 | 1 | 0 | 0 | 100.0% | — |
| VC211 | 1 | 0 | 0 | 100.0% | — |
| VC212 | 1 | 0 | 0 | 100.0% | — |
| VC213 | 1 | 0 | 0 | 100.0% | — |
| VC214 | 1 | 0 | 0 | 100.0% | — |
| VC215 | 5 | 0 | 0 | 100.0% | — |
| VC216 | 7 | 0 | 0 | 100.0% | — |
| VC217 | 4 | 0 | 0 | 100.0% | — |
| VC218 | 2 | 0 | 0 | 100.0% | — |
| VC219 | 4 | 0 | 0 | 100.0% | — |
| VC220 | 7 | 0 | 0 | 100.0% | — |
| VC221 | 8 | 0 | 0 | 100.0% | — |
| VC222 | 4 | 0 | 0 | 100.0% | — |
| VC223 | 6 | 0 | 0 | 100.0% | — |
Only rules with at least one ground-truth entry in the corpus appear here.
Head-to-head vs free scanners
| VC Rule | XploitScan | Semgrep | Sem? | Bearer | Bear? |
|---|---|---|---|---|---|
| ENTROPY | 100.0% | 0.0% | ✗ | 14.3% | ✓ |
| VC001 | 100.0% | 0.0% | ✗ | 38.5% | ✓ |
| VC002 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC003 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC004 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC005 | 75.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC006 | 100.0% | 11.1% | ✓ | 44.4% | ✓ |
| VC007 | 100.0% | 40.0% | ✓ | 60.0% | ✓ |
| VC008 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC009 | 50.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC010 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC011 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC012 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC013 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC014 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC015 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC016 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC017 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC018 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC019 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC020 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC021 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC022 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC023 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC024 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC025 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC026 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC027 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC028 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC029 | 100.0% | 33.3% | ✓ | 100.0% | ✓ |
| VC030 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC031 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC032 | 100.0% | 60.0% | ✓ | 60.0% | ✓ |
| VC033 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC034 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC035 | 100.0% | 50.0% | ✓ | 100.0% | ✓ |
| VC036 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC037 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC038 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC039 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC040 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC041 | 100.0% | 16.7% | ✓ | 83.3% | ✓ |
| VC042 | 100.0% | 0.0% | ✗ | 10.0% | ✓ |
| VC043 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC044 | 85.7% | 14.3% | ✓ | 100.0% | ✓ |
| VC045 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC046 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC047 | 100.0% | 50.0% | ✓ | 50.0% | ✓ |
| VC048 | 100.0% | 0.0% | ✗ | 33.3% | ✓ |
| VC049 | 100.0% | 0.0% | ✗ | 33.3% | ✓ |
| VC050 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC051 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC052 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC053 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC054 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC055 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC056 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC057 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC058 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC059 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC060 | 100.0% | 50.0% | ✓ | 100.0% | ✓ |
| VC061 | 100.0% | 66.7% | ✓ | 66.7% | ✓ |
| VC062 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC063 | 100.0% | 0.0% | ✗ | 83.3% | ✓ |
| VC064 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC065 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC066 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC067 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC068 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC069 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC070 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC071 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC072 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC073 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC074 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC075 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC076 | 100.0% | 33.3% | ✓ | 0.0% | ✗ |
| VC077 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC078 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC079 | 100.0% | 100.0% | ✓ | 50.0% | ✓ |
| VC080 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC081 | 100.0% | 50.0% | ✓ | 100.0% | ✓ |
| VC082 | 100.0% | 40.0% | ✓ | 40.0% | ✓ |
| VC083 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC084 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC085 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC086 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC087 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC088 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC089 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC090 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC091 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC092 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC093 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC094 | 100.0% | 50.0% | ✓ | 80.0% | ✓ |
| VC095 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC096 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC097 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC098 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC099 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC100 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC101 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC102 | 66.7% | 0.0% | ✗ | 0.0% | ✗ |
| VC103 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC104 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC105 | 50.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC106 | 75.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC107 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC108 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC109 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC110 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC111 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC112 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC113 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC114 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC115 | 100.0% | 50.0% | ✓ | 0.0% | ✗ |
| VC116 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC117 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC118 | 100.0% | 28.6% | ✓ | 85.7% | ✓ |
| VC119 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC120 | 66.7% | 33.3% | ✓ | 0.0% | ✗ |
| VC121 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC122 | 100.0% | 0.0% | ✗ | 66.7% | ✓ |
| VC123 | 100.0% | 36.4% | ✓ | 0.0% | ✗ |
| VC124 | 100.0% | 8.3% | ✓ | 8.3% | ✓ |
| VC125 | 100.0% | 0.0% | ✗ | 75.0% | ✓ |
| VC126 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC127 | 100.0% | 0.0% | ✗ | 75.0% | ✓ |
| VC128 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC129 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC130 | 75.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC131 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC132 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC133 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC134 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC135 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC136 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC137 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC138 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC139 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC140 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC141 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC142 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC143 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC144 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC145 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC146 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC147 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC148 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC149 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC150 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC151 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC152 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC153 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC154 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC155 | 100.0% | 0.0% | ✗ | 25.0% | ✓ |
| VC156 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC157 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC158 | 100.0% | 0.0% | ✗ | 66.7% | ✓ |
| VC159 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC160 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC161 | 66.7% | 0.0% | ✗ | 33.3% | ✓ |
| VC162 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC163 | 50.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC164 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC165 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC166 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC167 | 66.7% | 0.0% | ✗ | 33.3% | ✓ |
| VC168 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC169 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC170 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC171 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC172 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC173 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC174 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC175 | 66.7% | 0.0% | ✗ | 33.3% | ✓ |
| VC176 | 50.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC177 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC178 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC179 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC180 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC181 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC182 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC183 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC184 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC185 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC186 | 100.0% | 50.0% | ✓ | 0.0% | ✗ |
| VC187 | 100.0% | 66.7% | ✓ | 0.0% | ✗ |
| VC188 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC189 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC190 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC191 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC192 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC193 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC194 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC195 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC196 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC197 | 100.0% | 33.3% | ✓ | 33.3% | ✓ |
| VC198 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC199 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC200 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC201 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC202 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC203 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC204 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC205 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC206 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC207 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC208 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC209 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC210 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC211 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC212 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC213 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC214 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC215 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC216 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC217 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC218 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC219 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC220 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC221 | 100.0% | 12.5% | ✓ | 12.5% | ✓ |
| VC222 | 100.0% | 25.0% | ✓ | 0.0% | ✗ |
| VC223 | 100.0% | 16.7% | ✓ | 50.0% | ✓ |
Methodology. All scanners run against the same 615-fixture labeled corpus. A VC rule counts as "covered" by another scanner if any of that scanner's rules fires within ±10 lines of our expected range in the correct file. We don't require rule-ID equivalence — the question is capability to detect the class of vulnerability, not taxonomy alignment.
Semgrep. Version 1.86.0 with community rulesets p/security-audit, p/owasp-top-ten, p/javascript, p/typescript, p/react. Semgrep's Pro rules — included free for up to 10 contributors with a Semgrep account — would likely score higher; we ran the open-source CLI with registry rulesets because it needs no account.
XploitScan. Scored with the full 223-rule catalogue — what the CLI, GitHub Action, MCP server and VS Code extension run with a paid plan's API key. Without one, each runs 30 of those rules and would score lower here. We say so because Semgrep and Bearer are scored as free, no-account runs, and a comparison that is free on one side only is not one.
Bearer. 1.50.0, SAST scanner mode, Bearer's built-in security ruleset. Free to use under the Elastic License 2.0 (source-available, not OSI open source); no account required. Bearer describes itself as a SAST tool covering both security and privacy risks.
FP counting. Both Semgrep and Bearer use their own rule taxonomies, so we can't per-rule attribute FPs against our mustNotFire list the way we can with our own scanner. Any finding from those scanners on a clean fixture counts as an FP — a stricter interpretation that puts the third-party scanners at a precision disadvantage we acknowledge. Scored the same way, with every finding on a clean fixture counted, XploitScan had 69 findings on 43 of the 128 clean fixtures in a September 24, 2026 re-run, for 75.6% precision and 85.8% F1.
The Vibe Security Index — the same scanner, pointed at real repositories
Everything above answers one question: is this scanner accurate? That is a question about the tool, and it can only be answered against code where the right answer is already known. The Vibe Security Index answers a different one — what does the code AI app builders actually ship look like? — by running this same deterministic rule set over public repositories generated by Lovable, v0, Bolt and Base44.
The two belong together. A vendor publishing a survey of other people's code is worth exactly the published, checkable accuracy of the tool that produced it, misses included. This page is what earns the right to publish that one.
Aggregate only: the Index never names a repository, an owner, or a URL. See what it found →
Spot a gap?
The corpus is open and the runner is in the repo. If you have a real-world vulnerability pattern that our scanner misses, open a PR with a fixture at xploitscan-benchmark/test-fixtures/ or email admin@xploitscan.com. See the disclosure policy for anything you need to keep private.