ShadowHunter

Enterprise-Grade Dark Web Threat Intelligence Platform

Monitor credentials, track threat actors, and protect your organization from cyber threats

Features • Installation • Quick Start • Modules • API • Contributing

---

🎯 Overview

ShadowHunter is a comprehensive dark web intelligence platform designed for cybersecurity professionals, threat hunters, and security operations teams. It provides automated monitoring of underground markets, leak sites, and threat actor channels to deliver actionable intelligence.

███████╗██╗  ██╗ █████╗ ██████╗  ██████╗ ██╗    ██╗
██╔════╝██║  ██║██╔══██╗██╔══██╗██╔═══██╗██║    ██║
███████╗███████║███████║██║  ██║██║   ██║██║ █╗ ██║
╚════██║██╔══██║██╔══██║██║  ██║██║   ██║██║███╗██║
███████║██║  ██║██║  ██║██████╔╝╚██████╔╝╚███╔███╔╝
╚══════╝╚═╝  ╚═╝╚═╝  ╚═╝╚═════╝  ╚═════╝  ╚══╝╚══╝ 
                                                  
██╗  ██╗██╗   ██╗███╗   ██╗████████╗███████╗██████╗ 
██║  ██║██║   ██║████╗  ██║╚══██╔══╝██╔════╝██╔══██╗
███████║██║   ██║██╔██╗ ██║   ██║   █████╗  ██████╔╝
██╔══██║██║   ██║██║╚██╗██║   ██║   ██╔══╝  ██╔══██╗
██║  ██║╚██████╔╝██║ ╚████║   ██║   ███████╗██║  ██║
╚═╝  ╚═╝ ╚═════╝ ╚═╝  ╚═══╝   ╚═╝   ╚══════╝╚═╝  ╚═╝

---

✨ Features

🔍 Threat Intelligence

• Credential Monitoring - Real-time detection of leaked credentials
• Ransomware Tracking - Monitor leak sites from major ransomware groups
• IAB Marketplace Intelligence - Track Initial Access Broker listings
• Stealer Log Analysis - Parse RedLine, Vidar, Lumma, Raccoon logs
• IOC Enrichment - VirusTotal, Shodan, AbuseIPDB integration

🌐 Multi-Source Collection

• Telegram Integration - Monitor threat actor channels
• Tor Network Scraping - Access dark web resources anonymously
• Paste Site Monitoring - Track Pastebin, Ghostbin, and more
• GitHub Leak Detection - Scan for exposed secrets in code
• Scheduled Site Monitoring - Automated .onion site change detection

🔐 Session & Credential Validation

• Session Hijacking Detection - Validate stolen cookies (Google, Microsoft)
• Honeypot Detection - AI-powered fake credential identification
• Content Integrity Verification - Detect AI-generated disinformation

🔎 OSINT Capabilities

• Email Intelligence - Breach lookup, domain validation, Gravatar
• Phone Lookup - Carrier detection, country identification
• Username Enumeration - Cross-platform social media search
• Domain Reconnaissance - WHOIS, DNS, subdomain discovery

🕷️ Advanced Reconnaissance

• Attack Surface Scanner - Subdomain, port, and tech fingerprinting
• Google Dorking Engine - 50+ dork templates for exposure detection
• Web Crawler - Async crawling with onion link mapping
• Secret Detection - Find exposed API keys, tokens, credentials

🔗 Blockchain Forensics

• Multi-Chain Support - Bitcoin, Ethereum, Solana, Zcash, BSC, Polygon
• Wallet Profiling - Risk scoring, entity identification
• Transaction Tracing - Follow the money across addresses
• Mixer/Ransomware Detection - Identify high-risk wallets

🤖 Multi-Agent Hunters

• 19 specialized hunters — Paste, Crypto, Attribution, AgenticProfiler, DACOAnalyst, Identity, EntropyMonitor, PipelineDefender, SupplyChainHunter, AIThreatHunter, InsiderThreatHunter, SyntheticIDHunter, PhysicalSecHunter, ExploitMarketHunter, SocialMediaHunter, GitHubHunter, APTHunter, and ThreatScoringEngine (meta).
• 10 intelligence areas — Supply chain (typosquatting, malicious packages), AI/LLM threats (API key leaks, jailbreaks, weaponized models), insider threat (recruitment, exfiltration), synthetic ID (deepfake, voice clone, KYC bypass), physical security (lockpick, RFID, IoT), exploit markets (CVE, zero-day), social correlation, GitHub secret/repo analysis, APT attribution.
• ThreatScoringEngine — Aggregates risk scores from all hunter results; unified threat scores per entity; prioritization and escalation (e.g. score > 0.85).
• Escape hatches — Cross-hunter triggering (e.g. PasteHunter → CryptoHunter on wallet; AIThreatHunter → AgenticProfiler on jailbreak).
• CLI: shadowhunter hunters [--events-file events.json] [--enhanced] — full pipeline and synthesis report; --enhanced uses IntelligentEventRouter, ResourcePool, CircuitBreaker, and condition-based escape hatches. See RUNBOOK §8.

📊 Alerting & Reporting

• Multi-Channel Alerts - Telegram, Discord, Slack, webhooks
• Neo4j Graph Database - Relationship mapping and analysis
• Export Formats - JSON, CSV, Markdown reports
• Rate Limiting & Deduplication - Smart alert management

---

🚀 Installation

Prerequisites

• Python 3.9+
• Neo4j 5.x (optional, for graph database)
• Tor (optional, for dark web access)

Quick Install

# Clone the repository
git clone https://github.com/yourusername/shadowhunter.git
cd shadowhunter

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Verify installation
python main.py info

Docker Install

# Build and run with Docker Compose
docker-compose up -d

# API available at http://localhost:8000
# API docs at http://localhost:8000/docs

---

⚡ Quick Start

First run: Copy .env.example to .env and set NEO4J_PASSWORD. Run ./scripts/first_run.sh for a guided setup and next steps. See RUNBOOK §0 for the recommended order of operations (install → env → Neo4j schema → hunters → search → API + dashboard).

CLI Usage

# Display available commands
shadowhunter --help

# Scan domains for credential leaks
shadowhunter scan --domain acmecorp.com --domain techco.com

# Parse stealer logs
shadowhunter parse ./stealer_logs/ -c 20

# Validate session cookies
shadowhunter validate parsed.json --allowed yourcompany.com

# Scan for data leaks
shadowhunter leaks -k "mycompany" -d mycompany.com --github-token TOKEN

# Attack surface discovery
shadowhunter surface example.com --deep

# Blockchain forensics
shadowhunter blockchain 0x742d35Cc6634C0532925a3b844Bc9e7595f5d5E2

# IOC enrichment
shadowhunter enrich 1.2.3.4 --virustotal YOUR_KEY

# AI content verification
shadowhunter verify "suspicious content here"

# Site monitoring scheduler
shadowhunter schedule --add http://site.onion --name "Site" -k breach

# Multi-channel alerts
shadowhunter alerts --test --telegram-token TOKEN

# Google dorking reconnaissance
shadowhunter dork example.com --category credentials --severity CRITICAL

# Web crawling
shadowhunter crawl https://target.com --depth 3 --output report.json

# OSINT lookups
shadowhunter osint email test@example.com
shadowhunter osint username johndoe
shadowhunter osint domain example.com

# Start continuous monitoring
shadowhunter monitor -d company.com --telegram --tor

# Multi-Agent Hunters (19 hunters, synthesis report)
shadowhunter hunters
shadowhunter hunters --events-file events.json
shadowhunter hunters --enhanced   # Priority routing, circuit breakers, condition-based escape hatches

shadowhunter search -q "credential leak" -o report.md   # Dark web search: Tor + LLM refine/summary

# Launch API server
shadowhunter api --port 8000

Python API

import asyncio
from shadowhunter_osint import OSINTEngine
from shadowhunter_dorking import DorkingEngine
from shadowhunter_crawler import ShadowCrawler

async def main():
    # OSINT Investigation
    osint = OSINTEngine()
    email_intel = await osint.investigate_email("target@company.com")
    print(f"Risk Level: {email_intel.risk_level}")
    print(f"Breaches: {len(email_intel.breaches)}")
    
    # Google Dorking
    dorking = DorkingEngine()
    report = await dorking.quick_scan("targetcompany.com")
    print(f"Critical findings: {report.critical_count}")
    
    # Web Crawling
    crawler = ShadowCrawler(max_depth=2, max_pages=50)
    crawl_report = await crawler.crawl("https://target.com")
    print(f"Emails found: {len(crawl_report.emails)}")
    print(f"Secrets detected: {len(crawl_report.secrets)}")

asyncio.run(main())

---

📦 Modules

Core Modules

Module	Description	File
Logger	Winston-style logging with colors, JSON output	shadowhunter_logger.py
Core Monitor	Credential and threat scanning	shadowhunter_core.py
Tor Scanner	Dark web access and scraping	shadowhunter_tor.py
Telegram Monitor	Channel monitoring via Telethon	shadowhunter_telegram.py
Neo4j Graph	Threat intelligence graph database	shadowhunter_neo4j.py

Intelligence Modules

Module	Description	Directory
Stealer Parser	Parse RedLine, Vidar, Lumma logs	shadowhunter/parsers/
IOC Enrichment	VirusTotal, Shodan, AbuseIPDB	shadowhunter/intelligence/
Session Validator	Google/Microsoft cookie validation	shadowhunter/validators/
Leak Monitor	GitHub, paste site secret scanning	shadowhunter/monitors/
Surface Scanner	Subdomain & port discovery	shadowhunter/scanner/
Blockchain Forensics	Multi-chain wallet analysis	shadowhunter/blockchain/

OSINT & Reconnaissance

Module	Description	File
Dorking Engine	Google dork-based reconnaissance	shadowhunter_dorking.py
Web Crawler	Async crawling, onion link mapping	shadowhunter_crawler.py
OSINT Engine	Email, phone, username, domain lookup	shadowhunter_osint.py

Advanced Features

Module	Description	File
Vanguard Verify	AI content integrity, honeypot detection	shadowhunter_verify.py
Site Scheduler	.onion monitoring, change detection	shadowhunter_scheduler.py
Multi-Channel Alerts	Telegram, Discord, Slack, webhooks	shadowhunter_alerts.py

API & Infrastructure

Module	Description	File
FastAPI Server	REST API backend	shadowhunter_api.py
Authentication	JWT auth with RBAC	shadowhunter_auth.py
Email Alerts	HTML email notifications	shadowhunter_email.py

---

🔧 Configuration

Environment Variables

# API Configuration
SHADOWHUNTER_API_HOST=0.0.0.0
SHADOWHUNTER_API_PORT=8000
SHADOWHUNTER_SECRET_KEY=your-secret-key

# Neo4j Database
NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=password

# Telegram API (for channel monitoring)
TELEGRAM_API_ID=your-api-id
TELEGRAM_API_HASH=your-api-hash
TELEGRAM_PHONE=+1234567890

# HaveIBeenPwned API
HIBP_API_KEY=your-hibp-api-key

# Google Custom Search (for dorking)
GOOGLE_API_KEY=your-google-api-key
GOOGLE_CSE_ID=your-cse-id

# Email Alerts
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=your-email@gmail.com
SMTP_PASSWORD=your-app-password

config.json

{
  "monitored_domains": [
    "acmecorp.com",
    "techcompany.com"
  ],
  "scan_intervals": {
    "telegram": 60,
    "tor": 21600,
    "pastebin": 300
  },
  "alert_thresholds": {
    "risk_score": 70,
    "credential_count": 10
  },
  "notification": {
    "email": true,
    "slack": false,
    "webhook": null
  }
}

---

🌐 API Reference

Authentication

# Register new user
curl -X POST http://localhost:8000/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"username": "analyst", "email": "analyst@company.com", "password": "SecurePass123!"}'

# Login and get token
curl -X POST http://localhost:8000/api/auth/login \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "username=analyst&password=SecurePass123!"

Endpoints

Method	Endpoint	Description
GET	/api/overview	Dashboard statistics
GET	/api/alerts	List alerts (filterable)
POST	/api/alerts	Create new alert
GET	/api/domains	List monitored domains
POST	/api/domains	Add domain to monitoring
GET	/api/threat-actors	List top threat actors
POST	/api/scan/start	Trigger manual scan
GET	/api/scan/status	Get scan status
GET	/api/export/report/{domain}	Export threat report

---

📊 Neo4j Queries

-- Find all threats for a domain
MATCH (d:Domain {domain: 'acmecorp.com'})-[r]-(threat)
RETURN d, type(r) as relationship, threat

-- Most active threat actors
MATCH (t:ThreatActor)-[r]->()
RETURN t.username, count(r) as activity
ORDER BY activity DESC
LIMIT 10

-- Attack chain analysis
MATCH path = (c:Credential)-[*1..3]-(d:Domain)
RETURN path
LIMIT 20

-- IAB to ransomware correlation
MATCH path = (iab:IABListing)-[:OFFERS_ACCESS_TO]->(d:Domain)<-[:VICTIMIZED]-(r:RansomwareGroup)
RETURN path

---

🖥️ API Server

ShadowHunter includes a FastAPI REST API for programmatic access:

Features:

• RESTful endpoints for all modules
• WebSocket support for real-time updates
• JWT authentication with RBAC
• Swagger/OpenAPI documentation
• Export capabilities (JSON, CSV)

Start the API:

shadowhunter api
# Or: python shadowhunter_api.py
# Access docs at: http://localhost:8000/docs

Web Dashboard

A minimal Next.js dashboard provides visualization for alerts, domains, threat actors, and analytics. Design follows Dieter Rams and Japanese minimalist principles: useful, understandable, unobtrusive.

Run the dashboard:

# Terminal 1: start API
shadowhunter api --port 8000

# Terminal 2: start dashboard
cd dashboard && npm install && npm run dev
# Open http://localhost:3000

Set NEXT_PUBLIC_API_URL=http://localhost:8000 in dashboard/.env.local if the API runs elsewhere. See dashboard/README.md for details.

Phase 3 Neo4j schema (optional)

For best graph performance with Phase 3 hunters, apply constraints and indexes once (Neo4j 5.x):

cypher-shell -u neo4j -p <password> < scripts/neo4j_schema_phase3.cypher

See scripts/README.md and RUNBOOK §8.

---

🛠️ Development

Project Structure

shadowhunter/
├── main.py                     # CLI entry point (hunters, api, scan, parse, enrich, ...)
├── requirements.txt            # Python dependencies
├── config.py                   # Configuration
├── config/                     # Hunter config (e.g. all_hunters.yaml)
├── scripts/                    # Neo4j schema (neo4j_schema_phase3.cypher — hunters node types)
├── tests/                      # Hunter and NL-to-Cypher tests
│
├── shadowhunter_*.py           # Core, Tor, Telegram, Neo4j, dorking, crawler, OSINT, etc.
├── shadowhunter_api.py         # FastAPI REST API
├── dashboard/                  # Next.js web dashboard (Rams/minimal UI)
│
└── shadowhunter/               # Intelligence & Phase 3
    ├── parsers/                # Stealer log parsing
    ├── intelligence/           # IOC extraction & enrichment
    ├── validators/             # Session validation
    ├── monitors/               # Leak monitoring
    ├── scanner/                # Attack surface scanning
    ├── blockchain/             # Crypto forensics
    ├── ingestion/              # Ingestion pipeline (I2P, STIX, Neo4j)
    ├── hunters/                # Multi-agent hunters (19 hunters)
    ├── orchestration/          # Orchestrator, EnhancedOrchestrator, escape hatches
    └── nl_to_cypher/           # Natural language to Cypher translation

Running Tests

# Install test dependencies
pip install pytest pytest-asyncio pytest-cov

# Run all tests
pytest tests/ -v

# Run with coverage
pytest tests/ --cov=shadowhunter --cov-report=html

---

🔐 Security & Ethics

Responsible Use

ShadowHunter is designed for defensive security purposes only. Users must:

1. Authorization: Only monitor domains and assets you own or have explicit permission to test
2. Legal Compliance: Follow all applicable laws including CFAA, GDPR, and local regulations
3. Data Handling: Treat discovered credentials and PII with appropriate security measures
4. Disclosure: Report discovered vulnerabilities through responsible disclosure channels

OpSec Considerations

• Use dedicated VMs for dark web research
• Enable Tor for .onion access
• Rotate exit nodes regularly
• Never store credentials in plaintext
• Encrypt all exports and reports

---

📌 Documentation map

Doc	Purpose
README.md	Overview, features, install, quick start (this file)
USAGE.md	Detailed usage for all commands and modules, including Phase 3 hunters
RUNBOOK.md	Ops: Docker, Neo4j, Redis, Phase 3 hunters, NL-to-Cypher, troubleshooting
CONTRIBUTING.md	How to contribute
dashboard/README.md	Next.js dashboard setup
scripts/README.md	Neo4j Phase 3 schema script

---

💡 Final thoughts

• Secrets: No production API keys or credentials in the repo; use environment variables or config (see Configuration). Social/GitHub adapters are interface-plus-example; wire real keys via env.
• Tests: Run pytest tests/ -v before release; add coverage for new hunters or NL-to-Cypher changes.
• Neo4j: Apply scripts/neo4j_schema_phase3.cypher once per database for best hunter graph performance.
• Roadmap: Extended orchestration (IntelligentEventRouter, ResourcePool, CircuitBreaker, EnhancedEscapeHatchRegistry) is implemented and available via shadowhunter hunters --enhanced; future work may deepen ingestion–orchestrator integration (e.g. Redis → hunters) and additional hunter specs from NEW ADDITIONS/.

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

🤝 Contributing

Contributions are welcome! Please read our Contributing Guidelines before submitting PRs.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

---

📞 Support

• Documentation: docs.shadowhunter.io
• Issues: GitHub Issues
• Discussions: GitHub Discussions

---

ShadowHunter — See beyond a shaded silhouette. One platform: ingest, hunt, score, graph.