Repository navigation
Wait for usage sharing records without spinning, and fill a message up to twice the minimum as .NET and Java do - #191
Merged
Conversation
The background thread waited for each record with the take timeout and did nothing when it ran out, so a take timeout of zero made it spin and use a whole processor core for as long as the element lived. It now waits with no time limit. A message is sent as soon as it holds the minimum entries, so the take timeout has no role here, and the builder says so.
The .NET and Java pipelines use the take timeout once a message holds the minimum entries per message, taking more entries until none arrives within it or the message holds twice the minimum. The usage sharing element now does the same, so a configuration behaves alike in each language and the take timeout has its effect again. Below the minimum the background thread still waits with no time limit, so an idle element uses no processor time whatever its take timeout. A new unit test queues five records with a minimum of two and a long take timeout, and checks the messages hold four and then one.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #154
Problem
The usage sharing element's background thread waited for each record with
recv_timeout(take_timeout), and when the wait ran out it did nothing and waited again (share_usage_element.rs L872-L904). WithShareUsageConfigBuilder::take_timeout(Duration::ZERO)every wait returned at once, so the thread spun and used a whole processor core for as long as the element lived. With the default of 100 milliseconds it woke ten times a second for nothing.The take timeout had no other effect, and each message held exactly the minimum entries per message. The .NET and Java elements use the take timeout once the minimum is queued, taking more entries until none arrives within it or the message holds twice the minimum (ShareUsageElement.cs L96-L100, ShareUsageElement.java L351-L359).
Change
recv, which has no time limit and returns an error only once the element has been dropped and every queued record has been received. An idle element therefore uses no processor time, whatever its take timeout.a_message_takes_up_to_twice_the_minimum_within_the_take_timeout, queues five records with a minimum of two and a long take timeout, and checks the messages hold four and then one.pipeline-engines-fiftyone/tests/idle_cpu.rs, leaves an element idle for a second with a take timeout of zero and of 100 milliseconds, and checks the process used less than 250 ms of processor time. It reads the process's processor time on Windows and Linux, and is not built elsewhere. It has a binary of its own so no other test's work is counted.The first commit of this pull request made the take timeout do nothing and sent exactly the minimum. The second gives the take timeout the role it has in .NET and Java. The specification is being amended to describe that rule (51Degrees/specifications#56).
How it was verified
On Windows 11 with Rust 1.95.0. The new tests were first run against the base commit 9fe201e, with nothing else changed, using
cargo test -p fiftyone-pipeline-core -p fiftyone-pipeline-engines-fiftyone --no-fail-fast -- --test-threads=1, and each failed as shown.idle_cpu.rscame after that run, so the failure was measured with a scratch test doing the same thing, an element left idle for two seconds. It used 1,953 ms of processor time with a take timeout of zero and 0 ms with the default of 100 ms.With the change,
rustfmt --checkpasses on the changed files. The workspace checks on a local merge of every pull request for 4.5.12 (cargo fmt --all -- --check,cargo clippy --workspace --all-targets -- -D warnings,cargo doc --workspace --no-depswith warnings denied, andcargo test --workspacewith the Device Detection Lite and IP Intelligence ASN data files) are added here when they finish.Behaviour that changes
What remains