Memory usage optimizations - 61% reduction

[digizeph]

Summary

This PR implements memory usage optimizations that reduce peak memory consumption by 61% when parsing large BGP table dumps, while maintaining clean code architecture with no duplication.

Changes

1. NLRI Add-Path Clone-Free Parsing (Major Optimization)

File: src/parser/utils.rs

Problem: The original implementation cloned the entire input buffer when the Add-Path heuristic triggered (first byte is 0), retaining it for potential retry. This caused ~2x memory usage during parsing.

Solution:

• Primary implementation: try_parse_prefix() - parses from a byte slice without consuming, returns (prefix, bytes_consumed)
• Convenience wrapper: read_nlri_prefix() - thin wrapper that calls try_parse_prefix and advances the cursor
• No code duplication: Single source of truth for prefix parsing logic
• Clone-free: Eliminates expensive input.clone() operation

Architecture:

// Primary: speculative parsing from slice
fn try_parse_prefix(data: &[u8], afi: &Afi, add_path: bool) 
    -> Result<(NetworkPrefix, usize), ParserError>

// Wrapper: for Buf trait consumers  
fn read_nlri_prefix(&mut self, afi: &Afi, add_path: bool) 
    -> Result<NetworkPrefix, ParserError> {
    let (prefix, consumed) = try_parse_prefix(self.chunk(), afi, add_path)?;
    self.advance(consumed);
    Ok(prefix)
}

2. Attribute Vector Pre-allocation

File: src/parser/bgp/attributes/mod.rs

Estimate capacity from data size: remaining_bytes / 3 (minimum bytes per attribute), capped at 256.

3. RIS Live Vector Pre-allocation

File: src/parser/rislive/mod.rs

Calculate capacity from announcements + withdrawals before creating the vector.

Benchmark Results

Test Setup:

• Platform: macOS (MacBook Pro, Apple Silicon)
• Test file: RouteViews LINX RIB dump (rib.20240101.0000.bz2, 151MB compressed)
• Baseline: bgpkit-parser v0.15.0 (Homebrew)
• Measurement: /usr/bin/time -l peak memory footprint

Results:

Metric	Baseline	Optimized	Improvement
Peak Memory	2,032 MB	784 MB	-1,248 MB (61.3%)
Parse time	Similar	Similar	No regression

Code Quality Improvements

Before: ~140 lines of duplicated prefix parsing logic (in read_nlri_prefix and try_parse_prefix)
After: ~70 lines in try_parse_prefix only, read_nlri_prefix is a 4-line wrapper

Benefits:

• ✅ Single source of truth
• ✅ No code duplication
• ✅ Cleaner architecture
• ✅ Same 61% memory savings

Test Coverage

9 comprehensive NLRI tests covering:

• Basic Add-Path parsing
• Multiple prefixes with Add-Path
• IPv6 Add-Path
• Non-zero path_id
• Empty input
• Truncated data
• Heuristic triggering then retry
• Zero-prefix edge case

Backward Compatibility

✅ No breaking changes:

• No API changes
• No behavior changes (except improved memory efficiency)
• All 565 existing tests pass
• read_nlri_prefix maintains same interface

Verification

• All 565 tests pass
• No clippy warnings (cargo clippy --all-targets --all-features -- -D warnings)
• Memory benchmark: 61% reduction verified
• No performance regression
• Code review: simplify agent approved (no over-engineering)

Related

Addresses code review feedback from simplify agent on PR #278 - eliminated code duplication while maintaining optimization gains.

[codecov]

Codecov Report

❌ Patch coverage is 87.05882% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.52%. Comparing base (fd11e4e) to head (6f8ad2f).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/parser/utils.rs	86.50%	22 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #278      +/-   ##
==========================================
+ Coverage   90.48%   90.52%   +0.03%     
==========================================
  Files          84       84              
  Lines       15729    15835     +106     
==========================================
+ Hits        14233    14335     +102     
- Misses       1496     1500       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.