Releases: bgpkit/bgpkit-parser
v0.16.0
Breaking changes
BgpOpenMessage::sender_iprenamed tobgp_identifier: The field type remainsIpv4Addr(aliased asBgpIdentifier), but the name now correctly reflects RFC 4271 terminology — this is the BGP Identifier, not necessarily the sender's IP address. (thanks @ties for the contribution)
Security improvements
- AS_PATH segment count validation: Added bounds check to prevent buffer over-read when parsing AS_PATH segments. Validates that
count * asn_len.bytes()does not exceed remaining buffer size (CWE-126). - BGP message length underflow protection: Added saturating arithmetic to prevent integer underflow when calculating message length (CWE-191).
- NLRI prefix length validation: Enforce maximum prefix lengths (32 bits for IPv4, 128 bits for IPv6) immediately after reading length byte (CWE-20).
- BGP marker RFC 4271 compliance: Fixed marker encoding to use 16 bytes of 0xFF as required by RFC 4271. Added validation on parse with warning for non-compliant markers.
- Communities divisibility validation: Validate that Communities (4 bytes), Extended Communities (8 bytes), and Large Communities (12 bytes) attribute lengths are properly divisible by their respective sizes.
- FlowSpec traffic rate validation: Added NaN and infinity checks for FlowSpec Traffic Rate extended communities to prevent undefined behavior in downstream calculations.
- FlowSpec DSCP byte offset: Corrected Traffic Marking extended community parsing per RFC 5575 (DSCP at proper byte offset).
- Production code safety: Replaced
assert_eq!withdebug_assert_eq!in production parsing code to prevent panics on malformed input. - Timestamp overflow handling: Fixed timestamp truncation for values beyond 2106 (u32::MAX) using proper bounds checking.
- Encoding truncation checks: Added overflow warnings for
as u8/as u16casts in encoding functions. - IPv4 enforcement for AGGREGATOR/ORIGINATOR_ID: Encoders now reject IPv6 addresses for these attributes (BGP specifications require IPv4 only).
New features
-
WebAssembly (WASM) support (experimental): New
wasmfeature flag compiles the BMP/BGP/MRT parsing core to WebAssembly for use in JavaScript environments. Published as@bgpkit/parseron npm with support for Node.js (CommonJS), bundlers (ES modules), and browsers/workers (ES modules with manual init). This feature is experimental and the API may change in future releases.- Core parsing functions (all platforms):
parseOpenBmpMessage(data): parses OpenBMP-wrapped BMP frames (e.g. RouteViews Kafka stream)parseBmpMessage(data, timestamp): parses raw BMP frames (without an OpenBMP header)parseBgpUpdate(data): parses a single BGP UPDATE message intoBgpElem[]parseMrtRecords(data): streaming generator that yields MRT records one at a time from a decompressed bufferparseMrtRecord(data)/resetMrtParser(): low-level MRT record parsing
- Node.js I/O helpers:
streamMrtFrom(pathOrUrl): fetch, decompress (gz/bz2), and stream-parse MRT records from a URL or local fileopenMrt(pathOrUrl): fetch and decompress MRT data into aBuffer
- Supports gzip (RIPE RIS) and bzip2 (RouteViews, requires optional
seek-bzipdependency) compression - Multi-target build script (
src/wasm/build.sh) produces nodejs, bundler, and web targets in a single npm package - JS wrapper handles JSON deserialization; TypeScript types included
- Node.js examples in
examples/wasm/: Kafka OpenBMP stream consumer and MRT file parser - Browser-based MRT explorer demo: mrt-explorer.labs.bgpkit.com
- Core parsing functions (all platforms):
-
BgpElem::peer_bgp_idfield:BgpElemnow exposes an optionalpeer_bgp_id: Option<BgpIdentifier>containing the peer's BGP Identifier (Router ID) when available. Populated from the PEER_INDEX_TABLE in TableDumpV2/RIB records;Nonefor BGP4MP records. -
with_filtersandadd_filtersmethods: Added new methods toBgpkitParserfor passing pre-builtVec<Filter>directly, addressing issue #271. (thanks @JustinLoye for the feature suggestion!)with_filters(filters: &[Filter])- replaces all existing filters with the provided slice; clones internally so no manual clone neededadd_filters(filters: &[Filter])- extends existing filters with the provided slice- Both methods enable building filter specifications independently and reusing them across multiple parsers without string parsing overhead
- Added rust documentation to
add_filtermethod with complete list of available filter types
Example:
use bgpkit_parser::BgpkitParser; use bgpkit_parser::parser::Filter; let filters = vec![ Filter::new("peer_ip", "185.1.8.65").unwrap(), Filter::new("type", "w").unwrap(), ]; // Reuse across multiple parsers let parser1 = BgpkitParser::new(url1).unwrap().with_filters(&filters); let parser2 = BgpkitParser::new(url2).unwrap().with_filters(&filters);
-
Immutable Elementor API: New methods enable parallel processing of MRT records
Elementor::with_peer_table(peer_table)creates a pre-initialized ElementorElementor::record_to_elems_iter(&self, record)returns a lazy iterator without requiring&mut selfBgpkitParser::into_elementor_and_raw_record_iter()returns(Elementor, iterator)with PeerIndexTable pre-extractedBgpkitParser::into_elementor_and_record_iter()returns(Elementor, iterator)with PeerIndexTable pre-extracted- New
RecordElemIterandBgpUpdateElemItertypes for lazy element iteration - New
ElemErrortype for explicit error handling - Parallel processing example demonstrating multi-threaded parsing with significant speedup
Performance improvements
-
Memory usage reduction (61% improvement): Optimized memory allocation patterns resulting in significant peak memory reduction:
- NLRI Add-Path clone-free parsing: Replaced input buffer cloning with speculative byte-slice parsing. The new
try_parse_prefixfunction parses from a byte slice without consuming, returning(prefix, bytes_consumed). Theread_nlri_prefixmethod is now a thin wrapper that callstry_parse_prefixand advances the cursor. This eliminates the expensiveinput.clone()operation while maintaining correct Add-Path heuristic retry behavior. - Attribute vector pre-allocation: Estimate capacity from data size (
remaining / 3bytes per attribute) instead of fixed 20, reducing reallocations for BGP messages with many attributes. - RIS Live vector pre-allocation: Calculate capacity from announcements + withdrawals counts before allocation, preventing growth reallocations.
- Measured improvement: Peak memory reduced from 2,037 MB to 789 MB (61.3% reduction) when parsing full BGP table dump (RouteViews LINX RIB, 151MB compressed).
- Code quality: Single implementation of prefix parsing logic in
try_parse_prefix, eliminating duplication withread_nlri_prefix.
- NLRI Add-Path clone-free parsing: Replaced input buffer cloning with speculative byte-slice parsing. The new
-
Use zerocopy for MRT header parsing
- Replaced manual byte parsing with zerocopy's
FromBytestrait forRawMrtCommonHeaderandRawMrtEtCommonHeader - Reduces bounds checking overhead by using compile-time verified struct layouts
- Added compile-time assertions to ensure header sizes match wire format (12 bytes standard, 16 bytes ET)
- Encoding now uses zerocopy's
IntoBytestrait for efficient byte conversion
- Replaced manual byte parsing with zerocopy's
-
Use zerocopy for BGP OPEN message and capability parsing
RawBgpOpenHeader(10 bytes),RawMultiprotocolExtensions(4 bytes), andRawFourOctetAs(4 bytes) use zerocopy struct layouts- Replaces sequential cursor reads with single bounds-checked struct references
Code improvements
- Enhanced warning messages: Added context to 14 warning messages across BGP, BMP, and MRT parsers to help identify which parsing stage encountered an issue. Also fixed typo in NLRI warning ("NRLI" → "NLRI"). (thanks @ties for the contribution)
- Codecov configuration: Added
require_base: trueto ensure Codecov only posts coverage reports when a valid base commit coverage exists, preventing comparisons against outdated baselines.
Bug fixes
- WASM
only_to_customerserialization: Added test to verify that messages without the OTC (Only To Customer) attribute correctly serializeonly_to_customerasnull(not0) in JSON output. This ensures JavaScript consumers can properly distinguish between "no OTC attribute" (null) and "OTC with AS 0" (0). - OTC attribute on withdrawal messages: Withdrawal elements no longer inherit the
only_to_customervalue from their associated UPDATE message. Since the OTC attribute (RFC 9234) is used for route leak detection on route advertisements, it is not semantically meaningful for withdrawals. Withdrawal elements now correctly haveonly_to_customer: null. - Fix
Attributes::from_iterforAttributeValueto apply default BGP attribute flags instead of empty flags- Prevents malformed encoding for well-known mandatory attributes such as
ORIGINandAS_PATH
- Prevents malformed encoding for well-known mandatory attributes such as
- Preserve add-path
path_idwhen encoding TABLE_DUMP_V2 RIB entries and emit add-path RIB subtypes fromMrtRibEncoderwhen entries carry path IDs - Encode BGP4MP ASNs using the subtype-selected ASN width and always refresh MRT header lengths during record encoding
Testing
- Added benchmarks for
into_raw_record_iterfor both updates and RIB dumps
v0.15.0
Breaking changes
- Negative Filter Syntax Change: Negative filters now use value-based negation instead of type-based negation
- Old syntax (v0.14.0):
.add_filter("!origin_asn", "13335")- no longer supported - New syntax:
.add_filter("origin_asn", "!13335")- prefix the value with! - For multi-value filters, prefix each value:
.add_filter("origin_asns", "!13335,!15169") - Mixing positive and negative values in the same filter is not allowed and will return an error
- CLI syntax remains the same:
--filter "origin_asn!=13335"(internally converted to value-based negation)
- Old syntax (v0.14.0):
New features
- OR Logic Filters: New filter types that accept comma-separated values with OR logic
origin_asns: Match elements from ANY of the specified origin ASNsprefixes: Match elements for ANY of the specified prefixes (also supportsprefixes_super,prefixes_sub,prefixes_super_sub)peer_asns: Match elements from ANY of the specified peer ASNspeer_ips: Now validates input and requires at least one IP address- Example:
.add_filter("origin_asns", "13335,15169,8075")matches elements from Cloudflare, Google, or Microsoft - All multi-value filters support negation with
!prefix on each value - New example
multiple_filters_or_logic.rsdemonstrating OR logic and negative filters
v0.14.0
New features
-
Negative Filter Support: Most filters now support negation by prefixing the filter type with
!!origin_asn: Match elements where origin AS is NOT the specified value!prefix: Match elements where prefix is NOT the specified value!peer_ip,!peer_asn,!type,!as_path,!community,!ip_version: All support negation- Note: Timestamp filters (
ts_start,ts_end) do not support negation - Example:
.add_filter("!origin_asn", "13335")matches all elements NOT from AS 13335 - New
Filter::Negated(Box<Filter>)variant wraps any filter to invert its match result - CLI: New
--filter/-foption supports both positive and negative filter expressions- Positive:
--filter "origin_asn=13335" - Negative:
--filter "origin_asn!=13335" - Can be used multiple times:
--filter "peer_ip!=192.168.1.1" --filter "type!=w"
- Positive:
-
RPKI RTR Protocol Support: Add full support for the RPKI-to-Router (RTR) protocol
- New
models::rpki::rtrmodule with all PDU types: SerialNotify, SerialQuery, ResetQuery, CacheResponse, IPv4Prefix, IPv6Prefix, EndOfData, CacheReset, RouterKey, ErrorReport - New
parser::rpki::rtrmodule with parsing (parse_rtr_pdu,read_rtr_pdu) and encoding (RtrEncodetrait) - Support for both RTR v0 (RFC 6810) and v1 (RFC 8210)
- Comprehensive error handling with
RtrErrorenum - New example
rtr_client.rsdemonstrating RTR client implementation with ROA fetching and route validation
- New
v0.13.0
Breaking changes
RawMrtRecordfield renamed:raw_bytesis nowmessage_bytesto clarify it contains only the message body- Added new
header_bytesfield containing the raw header bytes as read from the wire - The
raw_bytes()method now returns complete MRT record bytes (header + body) without re-encoding
- Added new
New features
- Add MRT record debugging and raw bytes export capabilities
RawMrtRecordnow stores bothheader_bytesandmessage_bytesto enable exact byte-for-byte export without re-encoding- New methods on
RawMrtRecord:raw_bytes(),write_raw_bytes(),append_raw_bytes(),total_bytes_len() - Added
Displayimplementations forMrtRecord,CommonHeader, andMrtMessagefor human-readable debug output - New examples:
mrt_debug.rsandextract_problematic_records.rsdemonstrating debug features
- CLI now supports record-level output and multiple output formats
- New
--level(-L) option:elems(default) orrecordsto control output granularity - New
--format(-F) option:default,json,json-pretty, orpsv - Record-level output with
--level recordsoutputs MRT records instead of per-prefix elements - JSON output for records provides full structured data for debugging
- New
- Add
UpdateIteratorandFallibleUpdateIteratorfor iterating over BGP announcements (#250)- New
MrtUpdateenum supporting both BGP4MP UPDATE messages and TableDumpV2 RIB entries Bgp4MpUpdatestruct for BGP4MP UPDATE messages with metadata (timestamp, peer_ip, peer_asn)TableDumpV2Entrystruct for RIB dump entries (prefix with multiple RIB entries per peer)into_update_iter()andinto_fallible_update_iter()methods onBgpkitParser- Middle ground between
MrtRecordandBgpElemfor more efficient processing
- New
- Add
update_messages_iterexample demonstrating the new iterator
Testing and fuzzing
- Add cargo-fuzz harness and initial fuzz targets (mrt_record, bgp_message, parser)
- Add additional fuzz targets (BMP/OpenBMP, RIS Live, attributes, MRT header, NLRI); enable rislive feature for fuzzing
Bug fixes
- Add bounds checks throughout parsers to avoid overread/advance/split_to panics
- Handle invalid MRT BGP4MP_ET header length gracefully (reject ET records with on-wire length < 4)
- Use originated time instead of MRT header time for TableDumpV2 messages (#252 by @ties)
- added a new example study
rib_entries_age_studydemonstrating the difference between dump time and entry originated time
- added a new example study
Tooling and benchmarking
- Add hyperfine benchmarking script for bgpkit-parser
v0.12.0
Examples and benchmarking
- Added a new runnable example:
examples/parse_single_file_parallel.rs.- Parses RIB file using thread pool with batching and collecting. Two modes: worker-only parsing (default) or worker
parsing + element conversion. - Includes bottleneck diagnostics, tunable parameters (batch size, workers, etc), benchmarking, and remote file
downloading.
- Parses RIB file using thread pool with batching and collecting. Two modes: worker-only parsing (default) or worker
Breaking changes
- change
path_idfromu32toOption<u32>for proper null handlingNetworkPrefix::path_idfield now properly represents absence withNoneinstead of using0NetworkPrefix::new()andNetworkPrefix::encode()signatures updated accordinglyRibEntrynow storespath_idfor TABLE_DUMP_V2 messages with AddPath support- fixes issue #217
Minumum Supported Rust Version (MSRV)
We now set a minimum supported Rust version (MSRV) of 1.87.0.
Code improvements
New RFCs Supported
- added BGP Flow-Spec parsing support following RFC 8955 and RFC 8956
- added BGP Tunnel Encapsulation attribute parsing support following RFC 9012, RFC 5640, and RFC 8365
- added MRT geo-location extensions support following RFC 6397
- added comprehensive BGP Link-State parsing support following RFC 7752 and extensions
- add support for RFC 8654 Extended Messages
- implemented comprehensive BGP capabilities support for all IANA-defined capability codes with RFC support
- added RFC 8950 support for IPv4 NLRI with IPv6 next-hops
- added RFC 8950 Extended Next Hop capability parsing and encoding
- added RFC 9069 validation warnings for BMP Local RIB peer types
Performance improvements
- optimized BytesMut buffer allocation patterns for better memory efficiency
- replaced
BytesMut::with_capacity() + resize()withBytesMut::zeroed()for cleaner initialization - added capacity pre-allocation in ASN encoding to avoid reallocations
- replaced unnecessary BytesMut allocations with direct slice references in MRT header parsing
- added capacity pre-allocation in MRT record encoding for optimal buffer sizing
- fixed non-canonical PartialOrd implementation for ASN type to follow Rust idioms
- replaced
Iterator improvements
- added fallible iterator implementations for explicit error handling
- implemented
FallibleRecordIteratorthat returnsResult<MrtRecord, ParserErrorWithBytes> - implemented
FallibleElemIteratorthat returnsResult<BgpElem, ParserErrorWithBytes> - added
into_fallible_record_iter()andinto_fallible_elem_iter()methods onBgpkitParser - allows users to handle parsing errors explicitly instead of having them silently skipped
- maintains full backward compatibility with existing error-skipping iterators
- reorganized iterator implementations into structured
itersmodule withdefaultandfalliblesubmodules
- implemented
- added raw MRT record iteration support for record-by-record processing (PR #239, credits to @ties)
- added
RawRecordIteratoraccessible viainto_raw_record_iter()onBgpkitParser - separated MRT record reading from parsing for clearer error handling and improved throughput
- moved
RawRecordIteratorinto dedicatediters::rawmodule for better organization - added raw record parsing to
scan_mrtexample - added tests comparing
into_raw_record_iter()withrecord_iter()behavior
- added
- optimized duplicate BGP attribute detection using a fixed-size boolean array to track seen attributes (PR #237, credits to @ties)
- added boundary test cases for BGP attribute types
Bug fixes
- fixed TABLE_DUMP_V2 parsing to properly store path_id when AddPath is enabled
- previously path_id was read but discarded, now properly stored in
RibEntry
- previously path_id was read but discarded, now properly stored in
- fixed BGP OPEN message optional parameter parsing to support multiple capabilities per parameter
- changed
ParamValue::CapabilitytoParamValue::Capacities(Vec<Capability>)to properly handle RFC 5492 format - previously parser only read the first capability in each optional parameter, ignoring subsequent capabilities
- now correctly parses all capabilities within each optional parameter
- updated encoding logic to write all capabilities in the vector
- changed
- fixed RFC 9072 Extended Optional Parameters Length validation
- corrected extended parameter format validation (must have non-zero opt_params_len)
- fixed capability length parsing to always use 1-byte length per RFC 5492, not variable length
- ensures proper RFC 9072 compliance for extended OPEN message format
- improved RFC 9069 validation for BMP Local RIB peer types
- fixed multi-protocol capability check to specifically validate MULTIPROTOCOL_EXTENSIONS capability presence
- previously checked for any capabilities, now correctly validates the required capability type
- added
CorruptedBgpMessageerror type for better BMP parsing error handling- provides more specific error information when BGP messages within BMP are corrupted
Maintenance
- updated dependencies
- updated crate documentation and README
- removed outdated documentation files
v0.11.1
Bug fixes
- Fixed an issue where IPv6 NLRI next-hop addresses were not properly passed to
BgpElemobjects.
Maintenance
- Apply
cargo clippylints to the codebase - Add more tests to improve code coverage
v0.11.0
Breaking changes
- removing string format prefix like
lg:orecv6for large and extended communities- users who work with community value strings will have a unified experience for all community values
- users can still use SDK to match and check different types of community values
Highlights
- storing AS path filter regex object for reuse
- this will improve filter performance
- allow filter messages by community values
- by converting community values to string and do string comparison
- add
localflag to allow local-only processing- this removes dependencies like
reqwest
- this removes dependencies like
- support converting singleton AsSet to Vec
- when calling
to_u32_vec_opton AS path object, singleton AsSet will be treated as single ASN
- when calling
Bug fixes
- fixed a bug where RIS-live withdrawal messages were not properly parsed
Documentation
- add new RIS-Live async example
Maintenance
- update dependencies
- add
.envto gitignore - remote unnecessary life-time annotations
v0.10.11
Highlights
- Improved RIS Live message types and handling
- clean up dependencies
modelsfeature is removed, and all dependencies are now required dependencies for the cratemodelsmodule no-longer requires dependencies introduced by parser, likebytesfor encoding
- decouple
oneiofromparserfeatureoneiois no-longer a required dependency for the parser- users can now use
parserfeature withoutoneio(with--no-default-featuresspecified)- to create a new parser, one needs to user
BgpkitParser::from_readerfunction instead ofBgpkitParser::new
- to create a new parser, one needs to user
- downstream libraries that only utilize parser code (such as RIS Live parsing) no-longer have to depend on oneio
- improve RouteViews Kafka example by allowing wildcard topic subscription
Bug fixes
- fixed an issue where when merging
AS_PATHandAS4_PATHsegments, the parser incorrectly usesAS_PATHsegment
content- this issue may manifest as
AS_TRANS(AS23456) appearing in the AS path while the correct 32-bit ASN has been
sent inAS4_PATH
- this issue may manifest as
v0.10.11-beta.1
Highlights
- Improved RIS Live message types and handling
- clean up dependencies
modelsfeature is removed, and all dependencies are now required dependencies for the cratemodelsmodule no-longer requires dependencies introduced by parser, likebytesfor encoding
- decouple
oneiofromparserfeatureoneiois no-longer a required dependency for the parser- users can now use
parserfeature withoutoneio(with--no-default-featuresspecified)- to create a new parser, one needs to user
BgpkitParser::from_readerfunction instead ofBgpkitParser::new
- to create a new parser, one needs to user
- downstream libraries that only utilize parser code (such as RIS Live parsing) no-longer have to depend on oneio
v0.10.10
Highlights
- update
oneioto v0.17.0- now users can set env var
ONEIO_ACCEPT_INVALID_CERTS=trueto disable certificate validation, useful in some
environment where users do not manage certificates
- now users can set env var