Open datasets, collection scripts, and methodology behind our published research.
Website · All Research · License
AppSec Santa is an independent review and comparison platform covering 129+ application security tools across 10 categories including SAST, SCA, DAST, IaC Security, and more.
This repository contains everything needed to verify, reproduce, or build upon our published research — raw datasets, collection scripts, and aggregation code.
33 servers · 433 tools · 2 scanners · 27 YARA detections · ~80% false positive rate
Cisco mcp-scanner v4.3.0 · mcp-scan v2.0.1
Documentation · Published Article
6 LLMs · 89 prompts · 534 code samples · 6 SAST tools · 1,173 findings triaged
GPT-5.2 · Claude Opus 4.6 · Gemini 2.5 Pro · DeepSeek V3 · Llama 4 Maverick · Grok 4
Documentation · Published Article
65 tools · 5 health dimensions · GitHub + npm + PyPI + Docker Hub data
Recency · Activity · Releases · Community · Responsiveness
Documentation · Published Article
10,000 websites · Mozilla Observatory scoring · A+ to F grading
CSP · HSTS · X-Frame-Options · Referrer-Policy · X-Content-Type-Options · Redirection · X-XSS-Protection
Documentation · Published Article
Each study follows a three-stage pipeline — collect raw data from public sources, aggregate into scored datasets, and publish findings with full reproducibility.
┌─────────────────────────────────────────────┐
│ Data Collection │
│ │
Source APIs ──────────►│ GitHub API · npm · PyPI · Docker Hub │
LLM APIs ────────────►│ OpenRouter · SAST tool scans │
Target sites ────────►│ HTTP HEAD requests · DNS queries │
│ │
└──────────────────┬──────────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ Aggregation & Scoring │
│ │
│ Merge datasets · Compute health scores │
│ Validate findings · Generate distributions │
│ │
└──────────────────┬──────────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ Output │
│ │
│ Final JSON dataset · Published article │
│ │
└─────────────────────────────────────────────┘
- Python 3.10+
- Node.js 18+
Study-specific dependencies are listed in each study's README.
Found an issue with our data or methodology? Open an issue and we'll look into it.
This project is licensed under the MIT License.
Built by AppSec Santa — curated application security tools comparison.