Skip to content

feat: add concrete profiling guide for Kelora optimization#130

Open
dloss wants to merge 2 commits intomainfrom
claude/setup-kelora-profiling-CNznH
Open

feat: add concrete profiling guide for Kelora optimization#130
dloss wants to merge 2 commits intomainfrom
claude/setup-kelora-profiling-CNznH

Conversation

@dloss
Copy link
Owner

@dloss dloss commented Jan 18, 2026

Add comprehensive profiling guide in dev/profiling-guide.md with actionable
optimization strategies for improving Kelora's performance.

Key content:

  • Quick start commands for macOS (samply/Instruments) and Linux (flamegraph)
  • 6 known Kelora hotspots with optimization opportunities:
    • JSON parsing (30-50% runtime): simd-json, custom deserializer
    • Regex compilation (5-20%): caching, literal optimization
    • Rhai execution (10-40%): native filters, AST precompilation
    • String allocations (5-15%): Cow, string interning
    • Parallel overhead (0-30%): batch tuning, work-stealing
    • Format detection (1-5%): caching, heuristics
  • 10 concrete optimization ideas ranked by impact (high >20%, medium 5-15%, low <5%)
  • 6-step performance improvement workflow (baseline → profile → identify → optimize → verify → test)
  • 3 practical before/after code examples
  • Architecture-specific profiling guides (JSON, Rhai, parallel, parsers)
  • Common pitfalls and troubleshooting
  • Flamegraph interpretation with example diagnoses
  • CI integration example

This guide provides concrete, actionable advice for improving Kelora's
performance, not just profiling tool usage. Referenced from AGENTS.md
for developer discoverability.

Add comprehensive profiling guide in dev/profiling-guide.md with actionable
optimization strategies for improving Kelora's performance.

Key content:
- Quick start commands for macOS (samply/Instruments) and Linux (flamegraph)
- 6 known Kelora hotspots with optimization opportunities:
  * JSON parsing (30-50% runtime): simd-json, custom deserializer
  * Regex compilation (5-20%): caching, literal optimization
  * Rhai execution (10-40%): native filters, AST precompilation
  * String allocations (5-15%): Cow<str>, string interning
  * Parallel overhead (0-30%): batch tuning, work-stealing
  * Format detection (1-5%): caching, heuristics
- 10 concrete optimization ideas ranked by impact (high >20%, medium 5-15%, low <5%)
- 6-step performance improvement workflow (baseline → profile → identify → optimize → verify → test)
- 3 practical before/after code examples
- Architecture-specific profiling guides (JSON, Rhai, parallel, parsers)
- Common pitfalls and troubleshooting
- Flamegraph interpretation with example diagnoses
- CI integration example

This guide provides concrete, actionable advice for improving Kelora's
performance, not just profiling tool usage. Referenced from AGENTS.md
for developer discoverability.
Replace simple filter examples with diverse, realistic use cases that
better test Kelora's performance characteristics:

Workload improvements:
- Quick Start: regex filtering + field extraction (was: simple equality filter)
- Step 2 examples: 5 diverse workloads covering different subsystems
  * Regex-heavy filtering (tests regex compilation, caching)
  * Field extraction/enrichment (tests Rhai, field access, hashing)
  * Aggregation/metrics (tests state management, maps)
  * Parallel processing (tests threading, batching)
  * Format parsing (tests different parser paths)
- Architecture-specific profiling: realistic field manipulation examples
- CI integration: regex-based workload (was: simple filter)

Added "Recommended Profiling Workloads" section explaining:
- 5 categories of realistic use cases to profile
- Why each tests different parts of the system
- Note that simple filters are already optimized (native code paths)

Rationale: The simple `e.level == 'ERROR'` filter is already optimized
with native code paths. Profiling should focus on realistic workloads
that stress regex compilation, Rhai execution, field manipulation,
aggregation, and parallel processing - areas with optimization potential.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants