[jwrosewell]

Closes #197.

Problem

The cloud request engine counted every failed answer towards its recovery gate (cloud-request-engine/src/engine.rs lines 1223-1259). The gate belongs to the engine, so every visitor shares it, and with the default settings ten failures within 100 seconds stop every request reaching the cloud for 60 seconds.

Measured against cloud.51degrees.com on 26 October 2026, the data endpoint answers a user-agent of 16,000 bytes with 200, and one of 32,000 or 60,000 bytes with 404 and a plain text body. The web integration's JSON endpoint accepts a form body of up to 64 KB and passes its fields to the engine as evidence, so one visitor sending ten such requests opened a recovery period for everyone, and could keep it open.

Change

A failed answer counts towards the gate only when it says the service is in trouble, being a status of 500 or more, a 429, or a success status whose body could not be used. A request that did not complete, such as a timeout, counts as before. A status from 400 to 499 other than 429 answers the request's own content and leaves the gate as it was. The documentation of CloudRequestEngine and of the crate says which failures count.

The recovery mode section of the specification gives the gate's purpose as protecting a site when the cloud becomes slow or fails, and does not say which failures count. The 51Degrees .NET pipeline counts every failure, so this is a difference from it, made because a gate shared by every visitor must not be opened by what one visitor sends.

How it was verified

New tests in cloud-request-engine/tests/mock_client.rs, over a transport that answers as the service did above, with the default recovery settings:

• answers_to_one_visitors_own_content_leave_the_next_visitor_served. Ten requests with a 32,000 byte user-agent, then a visitor with an ordinary one. Before the change, the same scenario failed with the next visitor refused, sending requests to the cloud service is temporarily restricted due to recent failures; recovery resumes in 60.0s. It was run as a test on release/4.5.12 with this sweep's other fixes merged, none of which touch the gate.
• service_errors_still_open_a_recovery_period. Ten 503 answers still open a recovery period, as ten transport failures do in the existing recovery_mode_blocks_after_repeated_data_failures.

Commands, run locally on Windows, on the fix branches of this sweep merged together, with the Device Detection Lite and IP Intelligence ASN data files in 51DEGREES_DD_PATH and 51DEGREES_IPI_PATH:

cargo test --no-fail-fast -p fiftyone-cloud-request-engine -p fiftyone-fodid-cloud -p fiftyone-javascript-builder -p fiftyone-json-builder -p fiftyone-ip-intelligence-cloud -p fiftyone-pipeline-web -p fiftyone-pipeline-web-axum -p fiftyone-device-detection-cloud
cargo clippy --workspace --all-targets -- -D warnings
RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps
cargo fmt --all -- --check

The results of these runs are added here when they finish.

Behaviour that changes

• A 400 to 499 answer other than 429 no longer counts towards a recovery period. With a credential the service refuses, every request now reaches the service and fails at once, where after ten failures the engine used to stop sending for 60 seconds.
• Timeouts, connection failures, 5xx answers, 429 answers and unusable success answers open a recovery period as before.

What remains

• Whether the specification should say which failures count, so the languages agree.
• The checks on this pull request.