Skip to content

Releases: bgpkit/bgpkit-parser

v0.16.0

08 Apr 05:04
c964c69

Choose a tag to compare

Breaking changes

  • BgpOpenMessage::sender_ip renamed to bgp_identifier: The field type remains Ipv4Addr (aliased as BgpIdentifier), but the name now correctly reflects RFC 4271 terminology — this is the BGP Identifier, not necessarily the sender's IP address. (thanks @ties for the contribution)

Security improvements

  • AS_PATH segment count validation: Added bounds check to prevent buffer over-read when parsing AS_PATH segments. Validates that count * asn_len.bytes() does not exceed remaining buffer size (CWE-126).
  • BGP message length underflow protection: Added saturating arithmetic to prevent integer underflow when calculating message length (CWE-191).
  • NLRI prefix length validation: Enforce maximum prefix lengths (32 bits for IPv4, 128 bits for IPv6) immediately after reading length byte (CWE-20).
  • BGP marker RFC 4271 compliance: Fixed marker encoding to use 16 bytes of 0xFF as required by RFC 4271. Added validation on parse with warning for non-compliant markers.
  • Communities divisibility validation: Validate that Communities (4 bytes), Extended Communities (8 bytes), and Large Communities (12 bytes) attribute lengths are properly divisible by their respective sizes.
  • FlowSpec traffic rate validation: Added NaN and infinity checks for FlowSpec Traffic Rate extended communities to prevent undefined behavior in downstream calculations.
  • FlowSpec DSCP byte offset: Corrected Traffic Marking extended community parsing per RFC 5575 (DSCP at proper byte offset).
  • Production code safety: Replaced assert_eq! with debug_assert_eq! in production parsing code to prevent panics on malformed input.
  • Timestamp overflow handling: Fixed timestamp truncation for values beyond 2106 (u32::MAX) using proper bounds checking.
  • Encoding truncation checks: Added overflow warnings for as u8/as u16 casts in encoding functions.
  • IPv4 enforcement for AGGREGATOR/ORIGINATOR_ID: Encoders now reject IPv6 addresses for these attributes (BGP specifications require IPv4 only).

New features

  • WebAssembly (WASM) support (experimental): New wasm feature flag compiles the BMP/BGP/MRT parsing core to WebAssembly for use in JavaScript environments. Published as @bgpkit/parser on npm with support for Node.js (CommonJS), bundlers (ES modules), and browsers/workers (ES modules with manual init). This feature is experimental and the API may change in future releases.

    • Core parsing functions (all platforms):
      • parseOpenBmpMessage(data): parses OpenBMP-wrapped BMP frames (e.g. RouteViews Kafka stream)
      • parseBmpMessage(data, timestamp): parses raw BMP frames (without an OpenBMP header)
      • parseBgpUpdate(data): parses a single BGP UPDATE message into BgpElem[]
      • parseMrtRecords(data): streaming generator that yields MRT records one at a time from a decompressed buffer
      • parseMrtRecord(data) / resetMrtParser(): low-level MRT record parsing
    • Node.js I/O helpers:
      • streamMrtFrom(pathOrUrl): fetch, decompress (gz/bz2), and stream-parse MRT records from a URL or local file
      • openMrt(pathOrUrl): fetch and decompress MRT data into a Buffer
    • Supports gzip (RIPE RIS) and bzip2 (RouteViews, requires optional seek-bzip dependency) compression
    • Multi-target build script (src/wasm/build.sh) produces nodejs, bundler, and web targets in a single npm package
    • JS wrapper handles JSON deserialization; TypeScript types included
    • Node.js examples in examples/wasm/: Kafka OpenBMP stream consumer and MRT file parser
    • Browser-based MRT explorer demo: mrt-explorer.labs.bgpkit.com
  • BgpElem::peer_bgp_id field: BgpElem now exposes an optional peer_bgp_id: Option<BgpIdentifier> containing the peer's BGP Identifier (Router ID) when available. Populated from the PEER_INDEX_TABLE in TableDumpV2/RIB records; None for BGP4MP records.

  • with_filters and add_filters methods: Added new methods to BgpkitParser for passing pre-built Vec<Filter> directly, addressing issue #271. (thanks @JustinLoye for the feature suggestion!)

    • with_filters(filters: &[Filter]) - replaces all existing filters with the provided slice; clones internally so no manual clone needed
    • add_filters(filters: &[Filter]) - extends existing filters with the provided slice
    • Both methods enable building filter specifications independently and reusing them across multiple parsers without string parsing overhead
    • Added rust documentation to add_filter method with complete list of available filter types

    Example:

    use bgpkit_parser::BgpkitParser;
    use bgpkit_parser::parser::Filter;
    
    let filters = vec![
        Filter::new("peer_ip", "185.1.8.65").unwrap(),
        Filter::new("type", "w").unwrap(),
    ];
    
    // Reuse across multiple parsers
    let parser1 = BgpkitParser::new(url1).unwrap().with_filters(&filters);
    let parser2 = BgpkitParser::new(url2).unwrap().with_filters(&filters);
  • Immutable Elementor API: New methods enable parallel processing of MRT records

    • Elementor::with_peer_table(peer_table) creates a pre-initialized Elementor
    • Elementor::record_to_elems_iter(&self, record) returns a lazy iterator without requiring &mut self
    • BgpkitParser::into_elementor_and_raw_record_iter() returns (Elementor, iterator) with PeerIndexTable pre-extracted
    • BgpkitParser::into_elementor_and_record_iter() returns (Elementor, iterator) with PeerIndexTable pre-extracted
    • New RecordElemIter and BgpUpdateElemIter types for lazy element iteration
    • New ElemError type for explicit error handling
    • Parallel processing example demonstrating multi-threaded parsing with significant speedup

Performance improvements

  • Memory usage reduction (61% improvement): Optimized memory allocation patterns resulting in significant peak memory reduction:

    • NLRI Add-Path clone-free parsing: Replaced input buffer cloning with speculative byte-slice parsing. The new try_parse_prefix function parses from a byte slice without consuming, returning (prefix, bytes_consumed). The read_nlri_prefix method is now a thin wrapper that calls try_parse_prefix and advances the cursor. This eliminates the expensive input.clone() operation while maintaining correct Add-Path heuristic retry behavior.
    • Attribute vector pre-allocation: Estimate capacity from data size (remaining / 3 bytes per attribute) instead of fixed 20, reducing reallocations for BGP messages with many attributes.
    • RIS Live vector pre-allocation: Calculate capacity from announcements + withdrawals counts before allocation, preventing growth reallocations.
    • Measured improvement: Peak memory reduced from 2,037 MB to 789 MB (61.3% reduction) when parsing full BGP table dump (RouteViews LINX RIB, 151MB compressed).
    • Code quality: Single implementation of prefix parsing logic in try_parse_prefix, eliminating duplication with read_nlri_prefix.
  • Use zerocopy for MRT header parsing

    • Replaced manual byte parsing with zerocopy's FromBytes trait for RawMrtCommonHeader and RawMrtEtCommonHeader
    • Reduces bounds checking overhead by using compile-time verified struct layouts
    • Added compile-time assertions to ensure header sizes match wire format (12 bytes standard, 16 bytes ET)
    • Encoding now uses zerocopy's IntoBytes trait for efficient byte conversion
  • Use zerocopy for BGP OPEN message and capability parsing

    • RawBgpOpenHeader (10 bytes), RawMultiprotocolExtensions (4 bytes), and RawFourOctetAs (4 bytes) use zerocopy struct layouts
    • Replaces sequential cursor reads with single bounds-checked struct references

Code improvements

  • Enhanced warning messages: Added context to 14 warning messages across BGP, BMP, and MRT parsers to help identify which parsing stage encountered an issue. Also fixed typo in NLRI warning ("NRLI" → "NLRI"). (thanks @ties for the contribution)
  • Codecov configuration: Added require_base: true to ensure Codecov only posts coverage reports when a valid base commit coverage exists, preventing comparisons against outdated baselines.

Bug fixes

  • WASM only_to_customer serialization: Added test to verify that messages without the OTC (Only To Customer) attribute correctly serialize only_to_customer as null (not 0) in JSON output. This ensures JavaScript consumers can properly distinguish between "no OTC attribute" (null) and "OTC with AS 0" (0).
  • OTC attribute on withdrawal messages: Withdrawal elements no longer inherit the only_to_customer value from their associated UPDATE message. Since the OTC attribute (RFC 9234) is used for route leak detection on route advertisements, it is not semantically meaningful for withdrawals. Withdrawal elements now correctly have only_to_customer: null.
  • Fix Attributes::from_iter for AttributeValue to apply default BGP attribute flags instead of empty flags
    • Prevents malformed encoding for well-known mandatory attributes such as ORIGIN and AS_PATH
  • Preserve add-path path_id when encoding TABLE_DUMP_V2 RIB entries and emit add-path RIB subtypes from MrtRibEncoder when entries carry path IDs
  • Encode BGP4MP ASNs using the subtype-selected ASN width and always refresh MRT header lengths during record encoding

Testing

  • Added benchmarks for into_raw_record_iter for both updates and RIB dumps

v0.15.0

29 Jan 03:56
5eb9107

Choose a tag to compare

Breaking changes

  • Negative Filter Syntax Change: Negative filters now use value-based negation instead of type-based negation
    • Old syntax (v0.14.0): .add_filter("!origin_asn", "13335") - no longer supported
    • New syntax: .add_filter("origin_asn", "!13335") - prefix the value with !
    • For multi-value filters, prefix each value: .add_filter("origin_asns", "!13335,!15169")
    • Mixing positive and negative values in the same filter is not allowed and will return an error
    • CLI syntax remains the same: --filter "origin_asn!=13335" (internally converted to value-based negation)

New features

  • OR Logic Filters: New filter types that accept comma-separated values with OR logic
    • origin_asns: Match elements from ANY of the specified origin ASNs
    • prefixes: Match elements for ANY of the specified prefixes (also supports prefixes_super, prefixes_sub, prefixes_super_sub)
    • peer_asns: Match elements from ANY of the specified peer ASNs
    • peer_ips: Now validates input and requires at least one IP address
    • Example: .add_filter("origin_asns", "13335,15169,8075") matches elements from Cloudflare, Google, or Microsoft
    • All multi-value filters support negation with ! prefix on each value
    • New example multiple_filters_or_logic.rs demonstrating OR logic and negative filters

v0.14.0

23 Dec 03:56
ce7257b

Choose a tag to compare

New features

  • Negative Filter Support: Most filters now support negation by prefixing the filter type with !

    • !origin_asn: Match elements where origin AS is NOT the specified value
    • !prefix: Match elements where prefix is NOT the specified value
    • !peer_ip, !peer_asn, !type, !as_path, !community, !ip_version: All support negation
    • Note: Timestamp filters (ts_start, ts_end) do not support negation
    • Example: .add_filter("!origin_asn", "13335") matches all elements NOT from AS 13335
    • New Filter::Negated(Box<Filter>) variant wraps any filter to invert its match result
    • CLI: New --filter / -f option supports both positive and negative filter expressions
      • Positive: --filter "origin_asn=13335"
      • Negative: --filter "origin_asn!=13335"
      • Can be used multiple times: --filter "peer_ip!=192.168.1.1" --filter "type!=w"
  • RPKI RTR Protocol Support: Add full support for the RPKI-to-Router (RTR) protocol

    • New models::rpki::rtr module with all PDU types: SerialNotify, SerialQuery, ResetQuery, CacheResponse, IPv4Prefix, IPv6Prefix, EndOfData, CacheReset, RouterKey, ErrorReport
    • New parser::rpki::rtr module with parsing (parse_rtr_pdu, read_rtr_pdu) and encoding (RtrEncode trait)
    • Support for both RTR v0 (RFC 6810) and v1 (RFC 8210)
    • Comprehensive error handling with RtrError enum
    • New example rtr_client.rs demonstrating RTR client implementation with ROA fetching and route validation

v0.13.0

08 Dec 04:40
b9407e5

Choose a tag to compare

Breaking changes

  • RawMrtRecord field renamed: raw_bytes is now message_bytes to clarify it contains only the message body
    • Added new header_bytes field containing the raw header bytes as read from the wire
    • The raw_bytes() method now returns complete MRT record bytes (header + body) without re-encoding

New features

  • Add MRT record debugging and raw bytes export capabilities
    • RawMrtRecord now stores both header_bytes and message_bytes to enable exact byte-for-byte export without re-encoding
    • New methods on RawMrtRecord: raw_bytes(), write_raw_bytes(), append_raw_bytes(), total_bytes_len()
    • Added Display implementations for MrtRecord, CommonHeader, and MrtMessage for human-readable debug output
    • New examples: mrt_debug.rs and extract_problematic_records.rs demonstrating debug features
  • CLI now supports record-level output and multiple output formats
    • New --level (-L) option: elems (default) or records to control output granularity
    • New --format (-F) option: default, json, json-pretty, or psv
    • Record-level output with --level records outputs MRT records instead of per-prefix elements
    • JSON output for records provides full structured data for debugging
  • Add UpdateIterator and FallibleUpdateIterator for iterating over BGP announcements (#250)
    • New MrtUpdate enum supporting both BGP4MP UPDATE messages and TableDumpV2 RIB entries
    • Bgp4MpUpdate struct for BGP4MP UPDATE messages with metadata (timestamp, peer_ip, peer_asn)
    • TableDumpV2Entry struct for RIB dump entries (prefix with multiple RIB entries per peer)
    • into_update_iter() and into_fallible_update_iter() methods on BgpkitParser
    • Middle ground between MrtRecord and BgpElem for more efficient processing
  • Add update_messages_iter example demonstrating the new iterator

Testing and fuzzing

  • Add cargo-fuzz harness and initial fuzz targets (mrt_record, bgp_message, parser)
  • Add additional fuzz targets (BMP/OpenBMP, RIS Live, attributes, MRT header, NLRI); enable rislive feature for fuzzing

Bug fixes

  • Add bounds checks throughout parsers to avoid overread/advance/split_to panics
  • Handle invalid MRT BGP4MP_ET header length gracefully (reject ET records with on-wire length < 4)
  • Use originated time instead of MRT header time for TableDumpV2 messages (#252 by @ties)
    • added a new example study rib_entries_age_study demonstrating the difference between dump time and entry originated time

Tooling and benchmarking

  • Add hyperfine benchmarking script for bgpkit-parser

v0.12.0

17 Oct 19:47
2559b99

Choose a tag to compare

Examples and benchmarking

  • Added a new runnable example: examples/parse_single_file_parallel.rs.
    • Parses RIB file using thread pool with batching and collecting. Two modes: worker-only parsing (default) or worker
      parsing + element conversion.
    • Includes bottleneck diagnostics, tunable parameters (batch size, workers, etc), benchmarking, and remote file
      downloading.

Breaking changes

  • change path_id from u32 to Option<u32> for proper null handling
    • NetworkPrefix::path_id field now properly represents absence with None instead of using 0
    • NetworkPrefix::new() and NetworkPrefix::encode() signatures updated accordingly
    • RibEntry now stores path_id for TABLE_DUMP_V2 messages with AddPath support
    • fixes issue #217

Minumum Supported Rust Version (MSRV)

We now set a minimum supported Rust version (MSRV) of 1.87.0.

Code improvements

New RFCs Supported

  • added BGP Flow-Spec parsing support following RFC 8955 and RFC 8956
  • added BGP Tunnel Encapsulation attribute parsing support following RFC 9012, RFC 5640, and RFC 8365
  • added MRT geo-location extensions support following RFC 6397
  • added comprehensive BGP Link-State parsing support following RFC 7752 and extensions
  • add support for RFC 8654 Extended Messages
  • implemented comprehensive BGP capabilities support for all IANA-defined capability codes with RFC support
  • added RFC 8950 support for IPv4 NLRI with IPv6 next-hops
  • added RFC 8950 Extended Next Hop capability parsing and encoding
  • added RFC 9069 validation warnings for BMP Local RIB peer types

Performance improvements

  • optimized BytesMut buffer allocation patterns for better memory efficiency
    • replaced BytesMut::with_capacity() + resize() with BytesMut::zeroed() for cleaner initialization
    • added capacity pre-allocation in ASN encoding to avoid reallocations
    • replaced unnecessary BytesMut allocations with direct slice references in MRT header parsing
    • added capacity pre-allocation in MRT record encoding for optimal buffer sizing
    • fixed non-canonical PartialOrd implementation for ASN type to follow Rust idioms

Iterator improvements

  • added fallible iterator implementations for explicit error handling
    • implemented FallibleRecordIterator that returns Result<MrtRecord, ParserErrorWithBytes>
    • implemented FallibleElemIterator that returns Result<BgpElem, ParserErrorWithBytes>
    • added into_fallible_record_iter() and into_fallible_elem_iter() methods on BgpkitParser
    • allows users to handle parsing errors explicitly instead of having them silently skipped
    • maintains full backward compatibility with existing error-skipping iterators
    • reorganized iterator implementations into structured iters module with default and fallible submodules
  • added raw MRT record iteration support for record-by-record processing (PR #239, credits to @ties)
    • added RawRecordIterator accessible via into_raw_record_iter() on BgpkitParser
    • separated MRT record reading from parsing for clearer error handling and improved throughput
    • moved RawRecordIterator into dedicated iters::raw module for better organization
    • added raw record parsing to scan_mrt example
    • added tests comparing into_raw_record_iter() with record_iter() behavior
  • optimized duplicate BGP attribute detection using a fixed-size boolean array to track seen attributes (PR #237, credits to @ties)
    • added boundary test cases for BGP attribute types

Bug fixes

  • fixed TABLE_DUMP_V2 parsing to properly store path_id when AddPath is enabled
    • previously path_id was read but discarded, now properly stored in RibEntry
  • fixed BGP OPEN message optional parameter parsing to support multiple capabilities per parameter
    • changed ParamValue::Capability to ParamValue::Capacities(Vec<Capability>) to properly handle RFC 5492 format
    • previously parser only read the first capability in each optional parameter, ignoring subsequent capabilities
    • now correctly parses all capabilities within each optional parameter
    • updated encoding logic to write all capabilities in the vector
  • fixed RFC 9072 Extended Optional Parameters Length validation
    • corrected extended parameter format validation (must have non-zero opt_params_len)
    • fixed capability length parsing to always use 1-byte length per RFC 5492, not variable length
    • ensures proper RFC 9072 compliance for extended OPEN message format
  • improved RFC 9069 validation for BMP Local RIB peer types
    • fixed multi-protocol capability check to specifically validate MULTIPROTOCOL_EXTENSIONS capability presence
    • previously checked for any capabilities, now correctly validates the required capability type
  • added CorruptedBgpMessage error type for better BMP parsing error handling
    • provides more specific error information when BGP messages within BMP are corrupted

Maintenance

  • updated dependencies
  • updated crate documentation and README
  • removed outdated documentation files

v0.11.1

06 Jun 17:00
2711b95

Choose a tag to compare

Bug fixes

  • Fixed an issue where IPv6 NLRI next-hop addresses were not properly passed to BgpElem objects.
    • The fix ensures that the next_hop value is extracted from either NEXT_HOP or MP_REACHABLE_NLRI BGP
      attribute, and is correctly set in the BgpElem object.
    • Fixes issue #211
    • Thanks @m-appel for reporting this issue!

Maintenance

  • Apply cargo clippy lints to the codebase
  • Add more tests to improve code coverage

v0.11.0

26 Feb 19:35
a13f550

Choose a tag to compare

Breaking changes

  • removing string format prefix like lg: or ecv6 for large and extended communities
    • users who work with community value strings will have a unified experience for all community values
    • users can still use SDK to match and check different types of community values

Highlights

  • storing AS path filter regex object for reuse
    • this will improve filter performance
  • allow filter messages by community values
    • by converting community values to string and do string comparison
  • add local flag to allow local-only processing
    • this removes dependencies like reqwest
  • support converting singleton AsSet to Vec
    • when calling to_u32_vec_opt on AS path object, singleton AsSet will be treated as single ASN

Bug fixes

  • fixed a bug where RIS-live withdrawal messages were not properly parsed

Documentation

  • add new RIS-Live async example

Maintenance

  • update dependencies
  • add .env to gitignore
  • remote unnecessary life-time annotations

v0.10.11

27 Oct 15:25
5fc869d

Choose a tag to compare

Highlights

  • Improved RIS Live message types and handling
  • clean up dependencies
    • models feature is removed, and all dependencies are now required dependencies for the crate
    • models module no-longer requires dependencies introduced by parser, like bytes for encoding
  • decouple oneio from parser feature
    • oneio is no-longer a required dependency for the parser
    • users can now use parser feature without oneio (with --no-default-features specified)
      • to create a new parser, one needs to user BgpkitParser::from_reader function instead of BgpkitParser::new
    • downstream libraries that only utilize parser code (such as RIS Live parsing) no-longer have to depend on oneio
  • improve RouteViews Kafka example by allowing wildcard topic subscription

Bug fixes

  • fixed an issue where when merging AS_PATH and AS4_PATH segments, the parser incorrectly uses AS_PATH segment
    content
    • this issue may manifest as AS_TRANS (AS23456) appearing in the AS path while the correct 32-bit ASN has been
      sent in AS4_PATH

v0.10.11-beta.1

16 Oct 21:26
cfaa312

Choose a tag to compare

v0.10.11-beta.1 Pre-release
Pre-release

Highlights

  • Improved RIS Live message types and handling
  • clean up dependencies
    • models feature is removed, and all dependencies are now required dependencies for the crate
    • models module no-longer requires dependencies introduced by parser, like bytes for encoding
  • decouple oneio from parser feature
    • oneio is no-longer a required dependency for the parser
    • users can now use parser feature without oneio (with --no-default-features specified)
      • to create a new parser, one needs to user BgpkitParser::from_reader function instead of BgpkitParser::new
    • downstream libraries that only utilize parser code (such as RIS Live parsing) no-longer have to depend on oneio

v0.10.10

06 Aug 03:11
136bdff

Choose a tag to compare

Highlights

  • update oneio to v0.17.0
    • now users can set env var ONEIO_ACCEPT_INVALID_CERTS=true to disable certificate validation, useful in some
      environment where users do not manage certificates