Skip to content

Accelerate SRX address-book construction with an optional Rust extension - #377

Open
dantrapp wants to merge 6 commits into
google:masterfrom
dantrapp:perf/rust-address-engine
Open

dantrapp wants to merge 6 commits into
google:masterfrom
dantrapp:perf/rust-address-engine

Conversation

@dantrapp

Copy link
Copy Markdown

SRX address-book construction repeatedly calls Python subnet comparisons while scanning each network group's existing entries. This change adds an optional Rust extension that batches those comparisons and returns indices into the original Python objects.

Capirca continues to use the existing Python implementation when the extension is absent. Installing Capirca itself does not require Rust. The extension is built separately with python -m pip install ./native and requires Python 3.9+ and Rust 1.83+.

The insertion behavior is preserved: the first overlapping network determines the result, equal existing entries are replaced together, and later contained networks remain in place. Python retains address objects, comments, tokens, ordering, and address-book list identity. Scoped IPv6 addresses and custom network subclasses use the Python implementation.

Performance

Complete CLI invocations against upstream fcca84f5df6e8262d55ecca20635c166009c2c09, using the bundled policies, default options except --max_renderers=1, and an Apple M4:

Workload Upstream median With Rust Speedup
All bundled samples 15.8686 s 1.7551 s 9.04x
Large SRX/Palo Alto sample 14.6689 s 0.6051 s 24.24x
Small SRX sample 0.2501 s 0.2510 s No meaningful change

These measurements include process startup, imports, definition loading, parsing, rendering, and file writes. All 49 output files in the complete sample run matched byte-for-byte. Each workload has three measured runs per backend after warm-up, with alternating order and warm filesystem caches.

A separate five-round comparison with address optimization enabled includes a Python integer implementation of the same batching and comparisons. That control captures much of the improvement; Rust is another 1.22x faster across all samples and 2.20x faster on the large mixed-target sample. These are bundled fixtures, including synthetic large networks, rather than a production policy corpus.

Raw results, environment, and reproduction commands.

Validation

  • Local full suite: 1,725 passed with the extension; 1,718 passed and seven native-only tests skipped with it uninstalled.
  • Differential coverage against the original insertion method, including 36,000 seeded incoming networks, first-match replacement, duplicates, address-family boundaries, and object identity.
  • A renderer integration test verifies native dispatch and identical output.
  • Release wheel build, Rust formatting, Clippy with warnings denied, and the existing CI lint selections on changed Python files passed.
  • Cross-platform CI passed: full suites with and without the extension on Python 3.14 across Linux, macOS, and Windows, plus focused SRX tests on Python 3.9.

@google-cla

google-cla Bot commented Oct 10, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant