Merged
Conversation
- Add clap 4 for CLI argument parsing - Refactor main.rs: dispatch to serve (default) or sniff subcommand - Create src/cli.rs with Cli/Command enums - Create src/sniff/config.rs with SniffConfig (env + CLI args) - Add new deps: clap, async-trait, reqwest, zstd - Update .env.sample with sniff + AI provider config vars - 12 unit tests (7 CLI parsing + 5 config loading) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Create src/sniff/discovery.rs: LogSource, LogSourceType, discovery functions for system logs, Docker containers, and custom paths - Create src/database/repositories/log_sources.rs: CRUD for log_sources and log_summaries tables (follows existing alerts repository pattern) - Add log_sources and log_summaries tables to init_database() - Export docker module from lib.rs for reuse by sniff discovery - 14 unit tests (8 discovery + 6 repository) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Create src/sniff/reader.rs with LogReader async trait and LogEntry struct - FileLogReader: byte offset tracking, incremental reads, log rotation detection - DockerLogReader: bollard-based container log streaming with timestamp filtering - JournaldReader: journalctl subprocess (Linux-gated with #[cfg(target_os = "linux")]) - Add futures-util dependency for Docker log stream consumption - 10 unit tests covering read, incremental, truncation, empty lines, metadata Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Create src/sniff/analyzer.rs with LogAnalyzer trait - OpenAiAnalyzer: single client for OpenAI/Ollama/vLLM/any compatible API sends batched logs to /chat/completions, parses structured JSON response - PatternAnalyzer: fallback local analyzer using regex-free pattern matching detects error spikes, counts errors/warnings without external AI - LogSummary and LogAnomaly types with serialization support - JSON response parsing with graceful handling of partial LLM output - 16 unit tests (prompt building, JSON parsing, pattern analysis, serialization) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Create src/sniff/consumer.rs with LogConsumer - FNV hashing deduplication with configurable capacity (100k entries) - zstd compression (level 3) with timestamped archive files - File purge via truncation (preserves fd for syslog daemons) - Docker log purge via /var/lib/docker/containers/ JSON log truncation - Full consume pipeline: deduplicate → compress → purge → report stats - ConsumeResult tracks entries_archived, duplicates_skipped, bytes_freed - 13 unit tests (hashing, dedup, compression, purge, full pipeline) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Reporter: converts LogSummary/LogAnomaly into Alerts using existing AlertManager infrastructure (route_by_severity, NotificationChannel) - SniffOrchestrator: full discover → read → analyze → report → consume pipeline with continuous and one-shot modes - Wire up run_sniff() in main.rs to use SniffOrchestrator - Add events, rules, alerting, models modules to binary crate - 7 new tests (reporter: 5, orchestrator: 3) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- GET /api/logs/sources — list discovered log sources
- POST /api/logs/sources — manually add a custom log source
- GET /api/logs/sources/{path} — get a single source
- DELETE /api/logs/sources/{path} — remove a source
- GET /api/logs/summaries — list AI summaries (optional source_id filter)
- Register routes in configure_all_routes
- 7 tests covering all endpoints
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- CHANGELOG: document all sniff additions (discovery, readers, AI analysis, consumer, reporter, orchestrator, REST API, deps) - README: add log sniffing to key features, architecture diagram, project structure, CLI usage examples, REST API examples, and completed tasks list Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- install.sh: POSIX shell installer — detects Linux x86_64/aarch64, downloads from GitHub Releases, verifies SHA256, installs to /usr/local/bin - release.yml: GitHub Actions workflow — builds Linux binaries on tag push using cross, creates release with tarballs + checksums - README: add curl install one-liner to Quick Start Usage: curl -fsSL https://raw.githubusercontent.com/vsilent/stackdog/dev/install.sh | sudo bash Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add --ai-model flag to specify AI model (e.g. qwen2.5-coder:latest) - Add --ai-api-url flag to specify API endpoint URL - Recognize "ollama" as AI provider alias (maps to OpenAI-compatible client) - CLI args override env vars for model and API URL - Log AI model and API URL at startup for transparency Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add debug/trace logging across entire sniff pipeline: discovery, reader, analyzer, orchestrator, reporter - Respect user RUST_LOG env var (no longer hardcoded to info) - Improve LLM response JSON extraction to handle: markdown code fences, preamble text, trailing text - Include raw LLM response in trace logs for debugging parse failures - Show first 200 chars of failed JSON in error messages - Add 5 tests for extract_json edge cases Usage: RUST_LOG=debug stackdog sniff --once ... Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add --slack-webhook CLI flag to sniff command - Read STACKDOG_SLACK_WEBHOOK_URL env var (CLI overrides env) - Implement actual HTTP POST to Slack incoming webhook API - Build proper JSON payloads with serde_json (color-coded by severity) - Add reqwest blocking feature for synchronous notification delivery - Wire NotificationConfig through SniffConfig → Orchestrator → Reporter - Add STACKDOG_WEBHOOK_URL env var support - Update .env.sample with notification channel examples - Add 3 tests for Slack webhook config (CLI, env, override priority) Usage: stackdog sniff --once --slack-webhook https://hooks.slack.com/services/T/B/xxx # or via env: export STACKDOG_SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T/B/xxx Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- actions/cache v2 → v4 - actions/upload-artifact v2 → v4 - actions/download-artifact v2 → v4 - actions/checkout v2 → v4 - docker/build-push-action v1 → v6 (+ docker/login-action v3) - github/codeql-action/upload-sarif v1 → v3 Fixes: deprecated action versions causing workflow failures Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace deprecated actions-rs/* with dtolnay/rust-toolchain + cargo commands - Fix broken rustfmt/clippy steps (were using wrong action parameters) - Use Swatinem/rust-cache for simpler, faster dependency caching - Use ubuntu-latest runners instead of self-hosted - Add Docker Buildx for improved image builds - Trigger on PRs to main and dev (pushes :latest on every build) - Use npm ci for deterministic frontend installs - Add artifact retention-days: 1 to save storage Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- reqwest: use rustls-tls instead of native-tls (no OpenSSL dependency) - release.yml: build x86_64/aarch64-unknown-linux-musl (static binaries) - Dockerfile: update debian:buster-slim → bookworm-slim, drop libpq-dev Statically linked binaries work on any Linux distro including Alpine. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add .githooks/pre-commit that runs: - cargo fmt --all --check (strict, blocks commit) - cargo clippy (shows warnings, blocks on compile errors) Enable with: git config core.hooksPath .githooks Also applies cargo fmt --all to fix existing formatting across codebase. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove unused imports across 17 files - Prefix unused variables/fields with underscore - Implement Default trait instead of inherent default() methods (dedup, threat_scorer) - Implement FromStr trait instead of inherent from_str() methods (AiProvider, LogSourceType) - Replace Iterator::last() with next_back() on DoubleEndedIterator - Derive Default for EbpfLoader instead of manual impl - Remove useless .into() conversion in database connection - Wrap too-many-arguments functions with param structs (SniffArgs, CreateLogSummaryParams) - Replace filter_map(Some(...)) with map(...) in build_readers - Replace manual find loop with Iterator::find - Collapse nested if-let in signature_matcher - Gate program_to_tracepoint with cfg for Linux+ebpf only - Move api module to library crate, fix binary to import from library - Fix pre-existing test_get_alerts_empty to init database - Use std::io::Error::other() instead of Error::new(Other, e) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove unused imports in nftables.rs, quarantine.rs - Replace inherent to_string() with Display trait for NfTable - Remove needless borrows in .args() calls (iptables.rs, nftables.rs) - Prefix unused variables with underscore (response.rs) - Add error() getter to ResponseLog Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NfTable implements Display, so format! uses it directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove unused imports (AlertSeverity, AlertType, anyhow::Result, SyscallEvent, SyscallType) - Prefix unused variables with underscore - Fix EbpfLoader::default() test (returns value, not Result) - Fix KernelVersionTooLow fields to use String not integer literals - Remove unnecessary mut on enricher variables Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Introduce Sidebar component with Overview/Threats/Alerts/Containers links - Update App layout to include sidebar + main dashboard content - Add section anchors in Dashboard for sidebar navigation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Move logo into sidebar at 39x39 (30% larger than 30x30)\n- Remove "Stackdog Security Dashboard" header text\n- Add top bar with right-aligned actions menu button (...) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Handle GitHub /releases/latest 404 gracefully\n- Fallback to most recent release entry\n- Improve install usage examples to main branch URL Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
install: fallback when latest release is missing
- Make .env loading non-fatal in main\n- Print warning and continue with existing environment\n- Ensures stackdog --help works even with malformed .env Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- README: switch install script examples to main and v0.2.1\n- CHANGELOG: document non-fatal .env loading and installer fallback Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ich already contained host-built build-script binaries. Now musl build uses its own isolated target di
… main and preserve current Docker, sniff, and runtime fixes.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+11
to
+74
| name: Build & Test | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout sources | ||
| uses: actions/checkout@v2 | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install stable toolchain | ||
| uses: actions-rs/toolchain@v1 | ||
| - name: Install Rust toolchain | ||
| uses: dtolnay/rust-toolchain@stable | ||
| with: | ||
| toolchain: stable | ||
| profile: minimal | ||
| override: true | ||
| components: rustfmt, clippy | ||
| targets: x86_64-unknown-linux-musl | ||
|
|
||
| - name: Cache cargo registry | ||
| uses: actions/cache@v2.1.6 | ||
| with: | ||
| path: ~/.cargo/registry | ||
| key: docker-registry-${{ hashFiles('**/Cargo.lock') }} | ||
| restore-keys: | | ||
| docker-registry- | ||
| docker- | ||
|
|
||
| - name: Cache cargo index | ||
| uses: actions/cache@v2.1.6 | ||
| with: | ||
| path: ~/.cargo/git | ||
| key: docker-index-${{ hashFiles('**/Cargo.lock') }} | ||
| restore-keys: | | ||
| docker-index- | ||
| docker- | ||
| - name: Cache Rust dependencies | ||
| uses: Swatinem/rust-cache@v2 | ||
|
|
||
| - name: Install cross | ||
| run: cargo install cross --git https://github.com/cross-rs/cross | ||
|
|
||
| - name: Generate Secret Key | ||
| run: | | ||
| head -c16 /dev/urandom > src/secret.key | ||
| run: head -c16 /dev/urandom > src/secret.key | ||
|
|
||
| - name: Cache cargo build | ||
| uses: actions/cache@v2.1.6 | ||
| with: | ||
| path: target | ||
| key: docker-build-${{ hashFiles('**/Cargo.lock') }} | ||
| restore-keys: | | ||
| docker-build- | ||
| docker- | ||
|
|
||
| - name: Cargo check | ||
| uses: actions-rs/cargo@v1 | ||
| with: | ||
| command: check | ||
| - name: Check | ||
| run: cargo check | ||
|
|
||
| - name: Cargo test | ||
| if: ${{ always() }} | ||
| uses: actions-rs/cargo@v1 | ||
| with: | ||
| command: test | ||
| - name: Format check | ||
| run: cargo fmt --all -- --check | ||
|
|
||
| - name: Rustfmt | ||
| uses: actions-rs/toolchain@v1 | ||
| with: | ||
| toolchain: stable | ||
| profile: minimal | ||
| override: true | ||
| components: rustfmt | ||
| command: fmt | ||
| args: --all -- --check | ||
|
|
||
| - name: Rustfmt | ||
| uses: actions-rs/toolchain@v1 | ||
| with: | ||
| toolchain: stable | ||
| profile: minimal | ||
| override: true | ||
| components: clippy | ||
| command: clippy | ||
| args: -- -D warnings | ||
|
|
||
| - name: Run cargo build | ||
| uses: actions-rs/cargo@v1 | ||
| with: | ||
| command: build | ||
| args: --release | ||
| - name: Clippy | ||
| run: cargo clippy -- -D warnings | ||
|
|
||
| - name: Test | ||
| run: cargo test | ||
|
|
||
| - name: Build static release | ||
| env: | ||
| CARGO_TARGET_DIR: target-cross | ||
| run: cross build --release --target x86_64-unknown-linux-musl | ||
|
|
||
| - name: npm install, build, and test | ||
| - name: Build frontend | ||
| working-directory: ./web | ||
| run: | | ||
| npm install | ||
| if [ -f package-lock.json ]; then | ||
| npm ci | ||
| else | ||
| npm install | ||
| fi | ||
| npm run build | ||
| # npm test | ||
|
|
||
| - name: Archive production artifacts | ||
| uses: actions/upload-artifact@v2 | ||
| with: | ||
| name: dist-without-markdown | ||
| path: | | ||
| web/dist | ||
| !web/dist/**/*.md | ||
|
|
||
| # - name: Archive code coverage results | ||
| # uses: actions/upload-artifact@v2 | ||
| # with: | ||
| # name: code-coverage-report | ||
| # path: output/test/code-coverage.html | ||
| - name: Display structure of downloaded files | ||
| run: ls -R web/dist | ||
|
|
||
| - name: Copy app files and zip | ||
| - name: Package app | ||
| run: | | ||
| mkdir -p app/stackdog/dist | ||
| cp target/release/stackdog app/stackdog | ||
| cp -a web/dist/. app/stackdog | ||
| cp target-cross/x86_64-unknown-linux-musl/release/stackdog app/stackdog/ | ||
| cp -a web/dist/. app/stackdog/ | ||
| cp docker/prod/Dockerfile app/Dockerfile | ||
| cd app | ||
| touch .env | ||
| tar -czvf ../app.tar.gz . | ||
| cd .. | ||
| touch app/.env | ||
| tar -czf app.tar.gz -C app . | ||
|
|
||
| - name: Upload app archive for Docker job | ||
| uses: actions/upload-artifact@v2.2.2 | ||
| - name: Upload build artifact | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: artifact-linux-docker | ||
| name: app-archive | ||
| path: app.tar.gz | ||
| retention-days: 1 | ||
|
|
||
| cicd-docker: | ||
| name: CICD Docker | ||
| #runs-on: ubuntu-latest | ||
| runs-on: [self-hosted, linux] | ||
| needs: cicd-linux-docker | ||
| docker: |
Comment on lines
+75
to
+101
| name: Docker Build & Push | ||
| runs-on: ubuntu-latest | ||
| needs: build | ||
| steps: | ||
| - name: Download app archive | ||
| uses: actions/download-artifact@v2 | ||
| - name: Download build artifact | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: artifact-linux-docker | ||
| name: app-archive | ||
|
|
||
| - name: Extract app archive | ||
| run: tar -zxvf app.tar.gz | ||
| - name: Extract archive | ||
| run: tar -xzf app.tar.gz | ||
|
|
||
| - name: Display structure of downloaded files | ||
| run: ls -R | ||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Docker build and publish | ||
| uses: docker/build-push-action@v1 | ||
| - name: Login to Docker Hub | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| username: ${{ secrets.DOCKER_USERNAME }} | ||
| password: ${{ secrets.DOCKER_PASSWORD }} | ||
| repository: trydirect/stackdog | ||
| add_git_labels: true | ||
| tag_with_ref: true | ||
| #no-cache: true | ||
|
|
||
| - name: Build and push | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: . | ||
| push: true | ||
| tags: trydirect/stackdog:latest |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.