Skip to content

perf: fast-path long identical partial ratio inputs - #132

Open
arindamsikder wants to merge 1 commit into
rapidfuzz:mainfrom
arindamsikder:fix/partial-ratio-identical-input
Open

arindamsikder wants to merge 1 commit into
rapidfuzz:mainfrom
arindamsikder:fix/partial-ratio-identical-input

Conversation

@arindamsikder

Copy link
Copy Markdown

Summary

  • Return the full-length, 100-point alignment immediately for identical inputs longer than 64 characters.
  • Apply the shortcut in the shared implementation used by direct, alignment and cached scoring.
  • Cover empty/short inputs, the 64/65 boundary, 50,000-character separate buffers, iterator overloads, wide strings and cutoff handling.

Problem

The example in rapidfuzz/RapidFuzz#430 compares "hello" * 10000 with itself. The current C++ implementation evaluates shorter prefixes before reaching the full-length match. On this Linux/GCC 15.2 machine, the original direct, cached and alignment calls each exceeded a 15-second probe limit.

Solution

Check exact equality only when the lengths match and exceed 64. Keep the existing prefix/suffix search and tie ordering for nonidentical inputs; the public empty-input and above-100 cutoff guards remain in effect.

Performance tradeoff: this is not a universal speedup. Five alternating base/patch benchmark runs of 200,000 comparisons of 32-character strings differing at the first or last character showed medians of 1.07232 → 1.12651 seconds (+5.05%) and 1.09900 → 1.16506 seconds (+6.01%). The 50,000-character example completed in 0.00154 / 0.00166 / 0.00153 seconds for direct/cached/alignment, including scorer construction. These are single-machine measurements, not cross-platform guarantees.

Testing

All execution used a network-disabled Bubblewrap sandbox with no account credentials or home directories mounted. C++11, GCC 15.2.0, CMake 3.31.6, Catch2 2.13.10:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release   -DCMAKE_CXX_STANDARD=11 -DCMAKE_CXX_STANDARD_REQUIRED=ON   -DCMAKE_CXX_EXTENSIONS=OFF -DRAPIDFUZZ_BUILD_TESTING=ON   -DFETCHCONTENT_SOURCE_DIR_CATCH2=/catch2 -DFETCHCONTENT_FULLY_DISCONNECTED=ON
cmake --build build -j 2
ctest --test-dir build -C Release --output-on-failure -j 2
build/test/test_fuzz PartialRatioIdenticalInputs --reporter compact
build/test/test_fuzz --reporter compact
  • Full suite: 10/10 passed. Focused regression: 329 assertions; affected fuzz suite: 624 assertions.
  • New regression compiled against old implementation: timed out at 8 seconds; final implementation passed within the same cap. Repeated after final test expansion.
  • AddressSanitizer + UndefinedBehaviorSanitizer Debug affected suite: 624 assertions passed (-fsanitize=address,undefined -fno-omit-frame-pointer).
  • RAPIDFUZZ_ENABLE_LINTERS=ON with pinned project_options v0.26.2: affected test_fuzz builds and passes. The all-target strict build fails on five -Werror=null-dereference diagnostics at unchanged Levenshtein_impl.hpp:1124; the same target on pristine base produces the identical diagnostics. No warning flags were weakened.
  • External deterministic comparison: 9,246 output rows byte-identical on base and patch, including scores, alignment indices, cached results, cutoff boundaries, short binary-string corpus, 63/64/65/128 lengths, early/late differences and mixed wide/integer values.
  • git diff --check, added-line security scan and independent AI diff review passed. Standalone clang-format and other compiler/platform matrices were not available locally. Catch2 emits its existing CMake deprecation warning.

Related Issue

Related to rapidfuzz/RapidFuzz#430. This addresses the identical-input C++ case only, not general worst-case complexity, Python interrupt handling, or a downstream Python dependency update/release.

Implemented and validated with Hermes AI automation; the independent local review was also AI-based, not a human maintainer approval.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant