Repository navigation
Release 4.5.12 - #203
Merged
Merged
Release 4.5.12#203
Conversation
ci/run-performance-tests.ps1 recovered each figure by matching the example's printed output with a regular expression and wrote the summary files itself, so nothing checked them. The examples already write their own results JSON when given --json-output. The adapter now passes each example --json-output and hands the file to common-ci's steps/publish-performance-results.ps1, which checks that it exists, is valid JSON, has a HigherIsBetter or LowerIsBetter section and holds only numbers, then copies it to test-results/performance-summary/results_<Name>.json. This is what the Go, Java, Node, PHP and Python adapters do. The configuration names and the HigherIsBetter metric names are unchanged, so both graphs keep their history. A results file left by an earlier run is deleted first, and a missing common-ci checkout fails before the build. Refs #70
reqwest names the URL it requested in its errors, and that URL is the formatted one, whose query carries the licence keys. The service put the error into its own message, so each key reached the error a caller gets and the warning the scheduler logs. The message now takes the error without its URL, with its causes, and names the configured URL.
The cloud creates each 51Did afresh for the request that asked for it. The request engine's response cache stored the whole response body, so a visitor whose evidence matched an earlier visitor's was served the 51Did created for that earlier request, through the identifier engine that reads its answer from the stored body. The engine now leaves a response with a top-level fodid member out of the cache, and does not serve one it finds there, asking the cloud instead. A response without one, such as the answer to a request that carries no id.usage evidence, is cached as before. Refs #74
The cloud request engine joined each form field into its cache key as name=value ended by a line break, with nothing marking where a value ended. The web integration decodes the query string and the JSON endpoint's form body, so a query value can hold a line break followed by text shaped like another field. Such a request was given the key of a request that really carries both fields, and whichever came first decided the answer the other was served from the cache. Each name and each value is now written after its length in bytes, so two requests share a key only when their credential, property list and evidence are the same.
The delay hashed the time elapsed since an instant made a moment before, which is close to zero on every call, with the thread's id, so every process drew from a handful of values and instances started together polled together. Polls after the first added no delay at all. The delay is now drawn from a randomly keyed hasher and added to every poll, as the specification says.
The JavaScript builder read the device Promise and Fetch properties to decide whether the script it renders uses promises and the fetch API, and cleared a flag held on the element whenever a read failed. The element is shared by every request, and the read fails whenever one request has no device answer, as after a failed cloud request, so one such request turned promise and fetch support off for every later visitor until the process restarted, leaving fod.promise undefined in every script. The flag is gone. Each request reads its own values, and a request without one renders the script without promise and fetch support for that request alone.
…automatic updates on, and watch only the data file The watcher's callback held the registration that owned the watcher, so the registration, its engine and the data set the engine held, the open data file and the watcher's thread outlived the service. The callback now holds the registration weakly. Automatic updates are the specification's master switch for the watcher as well as for polling, and the watcher now refreshes the engine only for events that name the data file, from the modified time the file had when it was registered.
A refresh swapped in the new data set but kept everything the engine had taken from the old one. The Device Detection engine went on listing and reading only the old file's properties, under the old tier, so a Lite engine updated to an Enterprise file reported 22 properties, the tier Lite, and no value for the properties the Lite file lacks. The IP Intelligence engine went on reporting the old tier and publish date. Each engine now keeps the properties a detection reads with the data set they belong to, and replaces its property metadata and tier when a data file with others is loaded. The pipeline traits lend those by reference for as long as the engine lives, so each one replaced is kept rather than freed. A refresh records the IP Intelligence file's publish date, and refreshes run one at a time.
The specification gives the JSON builder's top-level errors member as a list of message strings, and the other languages, the cloud service and the client script all use that shape. The Rust builder wrote an object keyed by element instead. The client script reads errors as a list, so it dropped its own errors on such a payload, and carried a stored errors object from session storage into later page views as though it were an aspect. The builder now writes the messages as a list, in the order they were recorded.
…ter it all the same The specification says a failed update on start-up falls back to the local file, and fails only when there is none. The service returned before it registered the data file, so the file was never polled or watched, and both engine builders turned the error into a failed build, so an application could not start while the update server was unreachable or refusing requests. The service now logs the failure and registers the file all the same, and the builders return the engine, which has already loaded the local file.
The client-side endpoints answer a request whose processing recorded errors with Cache-Control no-cache, which the documentation says is so a faulty result is never cached. The response still carried the ETag, and the ETag is worked out from the evidence alone. A browser keeps a no-cache response and asks with that ETag before using it again, so once processing worked again for the same evidence the endpoint answered 304, and the browser went on using the faulty result, such as a script with no device data after one failed cloud request. The response to a request that recorded errors now carries no ETag, so a browser that kept it asks again without a validator and gets a full response. A request that matches the ETag of a good response still gets its 304, even while processing is failing.
When the cloud request engine's call failed it recorded its own error and left no response. The IP Intelligence cloud engine then added a second error, a configuration error saying the request engine returned no JSON, although nothing in the configuration was wrong. The device and 51Did cloud engines already treat a missing response as the request engine's own failure. The IP Intelligence cloud engine now gives empty data when the request engine ran but left no response, and keeps the configuration error for a pipeline with no request engine before it.
A second call to FlowData::process ran every element again and added their errors to those of the first run, so the error it returned repeated the first run's failures. The specification says a flow data can only be processed once, and .NET and Java both refuse a second call. It now runs no element and returns a configuration error.
The aggregate error FlowData::process returns said only how many errors there were, and it has no source, so anything that prints errors through Display, such as anyhow, never showed which element failed or why. The message now lists each error it carries, each with credentials removed.
The sequence element added one to the request's sequence with n + 1, so the largest i64 overflowed, which panics in any build with overflow checks. A sequence with no next integer is now treated like one that does not parse, as a new sequence of 1.
Adding the add timeout to the current instant panicked when the sum could not be represented, so Duration::MAX panicked on every request. A timeout too long to add to the clock now has no deadline, as with the standard library's own timeouts.
The background thread waited for each record with the take timeout and did nothing when it ran out, so a take timeout of zero made it spin and use a whole processor core for as long as the element lived. It now waits with no time limit. A message is sent as soon as it holds the minimum entries, so the take timeout has no role here, and the builder says so.
The adapter ran cargo with --config source.toml to build the examples against this tree. Cargo only uses a patch whose version is the one it would pick, and the examples' lock pins the published 4.5.8 crates while the manifests here say 4.5.2, so every patch went unused. Every nightly from 13 October 2026 compiled the 4.5.8 crates from crates.io and printed "patch ... was not used in the crate graph" for all 25 crates. ci/test-examples-against-source.ps1 already versions the crates above every release, moves the lock to the local copies, fails if any patched crate still comes from crates.io and puts everything back. It now takes the command to run, which defaults to the tests it ran before, and the adapter runs the two performance examples through it. The nightly checks out the full history because the script versions the crates from the release tags. It no longer restores examples/Cargo.lock before the comparison, because the run leaves the tree as it found it. Refs #70
dd::Results::process_device_id passed the native parser the bytes of the string slice and their length. The parser reads each profile id with atoi, which reads to the first byte that is not a digit whatever length it is given, so digits that followed the slice in memory joined the last profile id and its profile was not found. The id is now passed as a null terminated copy, as ipi::Results::process_ip passes an address. A test processes a device id from the front of a longer string. Closes #155.
CollectionItem was declared with no fields, so ProfilePercentage, which holds one ahead of its weighting, read raw_weighting from the first bytes of the item's data pointer rather than 32 bytes in on a 64 bit target. It now holds the data, handle and collection pointer the C structure holds, so the weighting fiftyoneDegreesResultsIpiGetValues returns reads correctly. A test compares that weighting with the weighted value collection's for every property. Closes #156.
With process exceptions suppressed, an element's error was only added to the flow data's errors, and the core had no logging at all, so a failure left no trace in any log. The specification says suppressed errors are added to the errors and logged. Each is now logged at debug level, as .NET logs them, with credentials removed.
The element built its blocking reqwest client on the thread that created it. Building one inside an async runtime panics in a debug build, and a web server usually creates its pipeline inside its runtime. The client is now built by the first send, on the element's background thread.
… cloud request engine does
…iles when it fails The service read the whole response into memory with a single call, which reqwest's blocking client limits to its timeout, 30 seconds by default, then decompressed it into a second buffer. A 2.5 GB download could not finish in 30 seconds on most links, and would have held about 9.7 GB in memory if it had. The body is now read as it arrives into a file beside the data file, with its MD5 taken as it is read, and decompressed into a second file that replaces the data file with a rename. Both files are removed when the update fails, where the old temporary file stayed.
The sequence element added one with checked_add and started a new sequence of 1 when there was no next integer. The .NET and Java pipelines add one with integer arithmetic that wraps, and the specification says only that the sequence is incremented, so the element now wraps too. A request whose sequence is the largest i64 gets the smallest one, which the JavaScript builder already renders as 1, as the specification says for a sequence of zero or less. Nothing panics either way.
The .NET and Java pipelines use the take timeout once a message holds the minimum entries per message, taking more entries until none arrives within it or the message holds twice the minimum. The usage sharing element now does the same, so a configuration behaves alike in each language and the take timeout has its effect again. Below the minimum the background thread still waits with no time limit, so an idle element uses no processor time whatever its take timeout. A new unit test queues five records with a minimum of two and a long take timeout, and checks the messages hold four and then one.
The specification lists a folder for the operational data copy, and .NET and Java set it with SetTempDirPath on the on-premise engine builder, defaulting to the system's temporary folder. Both on-premise engine builders now take temp_dir_path, with the same default. The engine keeps the folder, so the copy a refresh makes goes there too. A new test in each engine's operational copy tests builds an engine with automatic updates on and a named folder, and checks the copy is there after the build and after a refresh, and gone once the engine is dropped.
jwrosewell
marked this pull request as ready for review
October 27, 2026 14:43
GCC and Clang builds define fiftyoneDegreesMutexCreate, MutexClose, MutexLock and MutexUnlock as functions in both copies of common-cxx (the MSVC build makes them macros), so a binary linking both engines held two definitions and the linker kept one. ci/check-native-symbols.ps1 failed on Ubuntu and macOS for this. With the four names added to symbol_prefix.h it reports that no common-cxx name is shared between the two libraries.
wasm32-wasip1 has no threads, so spawning one panics, and a panic on WebAssembly aborts the whole test binary. The test is left out there, as the other thread tests in the workspace already are.
The same test as in device-detection-shared, which aborts the wasm32-wasip1 test binary because WebAssembly has no threads.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 4.5.12 of the 51Degrees Rust crates. It merges
release/4.5.12, which holds the 28 pull requests below, each reviewed on its own, intomain. Merging this publishes all 25 crates as 4.5.12.Changes
Data updates
Web integration and cloud
Pipeline core
Usage sharing
Native bindings
Continuous integration
Behaviour that changes on upgrade
ETagvalue changes once, so a browser's first revalidation after the upgrade gets a full response. After that the same evidence gives the sameETagin every process. A response to a request that recorded errors has noETag.errorsas a list of messages, as the specification says.InMemory, an on-premise engine reads a copy of its data file, made in the system's temporary folder unless the builder'stemp_dir_pathnames another, so it needs room there for one copy, two during a refresh.Specification
Every change was checked against the pipeline specification and the .NET and Java pipelines. Where the specification said nothing, these changes propose what it should say, with the matching .NET and Java issues.
How it was verified
On Windows 11 with Rust 1.95.0, on a local merge of the 28 pull requests in the order they were merged into
release/4.5.12, whose tree is identical torelease/4.5.12:cargo fmt --all -- --checkpassed.cargo clippy --workspace --all-targets -- -D warningspassed.cargo doc --workspace --no-depswithRUSTDOCFLAGS="-D warnings"passed.cargo test --workspace --no-fail-fast, with the Device Detection Lite and IP Intelligence ASN data files in51DEGREES_DD_PATHand51DEGREES_IPI_PATH, gave 1,070 passed, 0 failed and 10 ignored across 109 test binaries. The ignored tests need the network with a live resource or license key, or the IP Intelligence Enterprise data file.ci/test-examples-against-source.ps1 -Version 4.5.12, the check the publish job runs before publishing, with no keys or data file paths set, gave 98 passed, 0 failed and 11 ignored across 49 test binaries, and left the tree unchanged.On Ubuntu 24.04 (under WSL) with Rust 1.95.0, on a line-ending-clean checkout of the same tree,
cargo test --workspace --no-fail-fastwith the same data files gave 1,068 passed, 2 failed and 10 ignored across 109 test binaries. The two failures arethe_watcher_runs_only_with_automatic_updates_onandthe_watcher_ignores_other_files_in_the_folder, which failed only because the temporary folder was on a Windows drive mounted into Linux, where file change notifications are not delivered. The same test binary run with the temporary folder on the Linux file system passes all three of its tests.The
Pull Requestworkflow had run no jobs since 12 October 2026. On this pull request it ran on all four platforms and on WebAssembly, and found two faults that 4.5.11 already had, both now fixed on this branch:ci/check-native-symbols.ps1foundfiftyoneDegreesMutexCreate,MutexClose,MutexLockandMutexUnlockdefined by both copies of common-cxx. GCC and Clang builds define them as functions, where the MSVC build makes them macros. 82a50a3 adds them toip-intelligence-sys/src/symbol_prefix.h, and the check now finds no shared name.reads_from_several_threads_agree_and_read_onceindevice-detection-sharedand inip-intelligence-sharedspawns threads, which wasm32-wasip1 does not have, and the panic aborts the test binary. d6fd0f9 and ca1bf02 leave both tests out on WebAssembly, as the other thread tests in the workspace already are.With those three commits every check passes, including the build and tests on Windows, Ubuntu 22.04, Ubuntu 24.04 and macOS, and the tests under wasmtime. The publish job builds the examples against this tree on Linux again before it publishes, and a failure there stops the release.
Closes #70
Closes #74
Closes #149
Closes #150
Closes #151
Closes #152
Closes #153
Closes #154
Closes #155
Closes #156
Closes #157
Closes #159
Closes #160
Closes #161
Closes #162
Closes #163
Closes #164
Closes #165
Closes #166
Closes #167
Closes #168
Closes #169
Closes #170
Closes #171
Closes #172
Closes #183
Closes #192
Closes #197