[jwrosewell]

Closes #167.

Builds on #194, whose commits it carries, because both change how each engine keeps its loaded data set. Merge #194 first, and this pull request then shows its own change alone.

Problem

Every profile but InMemory reads the data file while its data set lives, through handles the native library opens without delete sharing (common-sys/vendor/common-cxx/file.c line 169), and both engines opened the data file itself. On Windows an update cannot rename the new file over a file open that way, so with automatic updates on, an update under Default, LowMemory or HighPerformance downloaded the whole file and failed with NewFileCannotRename, and the next poll did the same. The specification makes an operational data copy necessary for disk based operation with automatic updates on (data-updates.md lines 249 to 252), and .NET documents its temporary copy as required for automatic updates when the data is not all in memory. #167 has the details.

Change

• With automatic updates on and any profile but InMemory, each engine copies the data file into the temporary folder and reads the copy, so the data file itself is never held open. A refresh copies the new file the same way.
• Each copy is kept with the data set read from it and removed after that data set is freed. A detection's reader holds the data set, results freed first, so a copy goes when the last detection reading it is dropped, including one that outlived a refresh.
• When the copy cannot be made, the engine reads the data file itself, so a full temporary folder does not stop it building.

How it was verified

New tests:

• an_update_replaces_the_data_file_under_every_profile, in device-detection-onpremise/tests/operational_copy.rs with the Lite Hash file and in ip-intelligence-onpremise/tests/operational_copy.rs with the ASN file, updates a running engine from a local server under Default, Balanced, LowMemory, HighPerformance and InMemory, and checks the update succeeds, the engine still detects or looks up, and nothing is left behind once the engine is dropped.
• each_copy_goes_with_the_data_set_read_from_it (device-detection-onpremise/tests/operational_copy.rs) counts the copies while an early detection outlives three refreshes, after it is dropped, and after the engine is dropped.

Before the change, on a branch holding only the new tests, all three fail. Under Default each update fails with data update failed (NewFileCannotRename): error writing data file '...\51Degrees.hash': Access is denied. (os error 5), for the Device Detection engine and the IP Intelligence engine alike, and no copy is made (one copy for the data set loaded, left: 0, right: 1). After it, all three pass.

Locally on Windows, with all seven fixes of this sweep merged together (#175, #178, #182, #184, #194, #195 and #196), cargo test --no-fail-fast -p fiftyone-pipeline-engines -p fiftyone-device-detection-onpremise -p fiftyone-ip-intelligence-onpremise -p fiftyone-device-detection -p fiftyone-ip-intelligence -- --nocapture, with 51DEGREES_DD_PATH naming the Lite Hash file, Enterprise-HashV41.hash beside it, and 51DEGREES_IPI_PATH naming the ASN file, 51Degrees-LiteV41.ipi beside it, gives 130 passed, 0 failed and 2 ignored across 24 test binaries. With the fixes merged, cargo doc --workspace --no-deps with RUSTDOCFLAGS="-D warnings" passes, and rustfmt --check passes on every file the seven fixes change. cargo clippy --workspace --all-targets -- -D warnings on the merged fixes has not run yet. The Pull Request workflow, which builds and tests each pull request on every platform, is waiting for approval to run.

Behaviour that changes

With automatic updates on and any profile but InMemory, an engine needs free space in the temporary folder for a copy of its data file, two while a refresh loads a new one, and takes the time to copy the file when it is built and on each refresh. In return an update can replace the data file on Windows.

What remains

• The copy goes into std::env::temp_dir(). .NET and the specification let the caller choose the folder. A setting for it would be new public API, so it is left for a decision.
• A copy left by a process that was killed is not removed by later runs.