[hahn-kev-bot]

---

Important

Stacked on #404, which fixes restore against libpalaso 18.0.0; retarget to master once that merges.
Waiting on sillsdev/hgresume#31. Each pull request now asks for the rest of the bundle, and until that PR ships the server reads all of it into memory and holds it for as long as the client takes to download it.

AI summary

Summary

Over the resumable transport, a pull used to fetch the bundle in chunks sized to finish inside the 30 second timeout. Now it fetches it in as few requests as the server allows, writing the body to the bundle file as it arrives:

 Pull
   loop
-    ask for chunkSize bytes (sized to take ~7 s on this link)
-    buffer the whole chunk, then append it to the bundle file
+    nothing known yet             -> ask for 1 byte (brings back the bundle size so far)
+    server has more than we do    -> ask for all of it but the last byte (served at once)
+    caught up                     -> ask for the rest, streamed to the bundle file
+    "in progress" reply           -> probe again (it doesn't say how far the server got)
+    connection cut / stall > 30 s -> next request resumes at the end of the file
+    Cancel                        -> checked as each piece of the body arrives

• The 30 s timeout now bounds each wait for data, not the whole request, so a slow link that keeps delivering is never cut off.
• IApiServer gains an Execute overload that streams a 200 body to a Stream, with a progress callback.
• The progress line shows an ETA from the measured rate, updated at most once a second.
• Push is unchanged.
• Integration tests can run against a server already running on the machine (HGRESUME_URL, HGRESUME_REPO_ROOT). They add a 3 MB clone, and a clone through a proxy that cuts the connection four times.

Evidence

• Timing, against hgresume built from source (measured by the author of the second commit):
  Before: a pull of one small change took 22 s, and a 3 MB clone took 37 s.
  After: the small pull takes three quick requests, and the clone takes 9 s.
• Cancel during a long download:
  Before: Pull_UserCancelsMidBody_StopsAndNextPullResumesFromBytesReceived fails. The cancelled request runs to the end, and the next pull resumes at byte 1049399.
  After: it passes. The download stops half way, and the next pull resumes at byte 524700.
• "In progress" reply once caught up:
  Before: Pull_InProgressWhenCaughtUp_ProbesNext fails because the next request asks for the rest again (chunkSize 2146434248), which waits out the server's 14 s poll every time.
  After: it passes. The next request is a 1-byte probe.
• All 71 HgResume tests in LibChorus.Tests pass on net8.0.

Merge Danger

Door: two-way

The client change needs no protocol change, and reverting it restores chunked pulls. A bundle half-downloaded by the new client resumes fine under the old one, because both resume from the end of the file.

Blast Radius: every resumable pull

• Old PHP server: asking for the rest of the bundle sends a chunkSize of about 2 GB. The C# server limits that to what is left of the bundle. The old PHP server passes it to fread, which allocates the full size, over the 128M memory_limit. Production needs to be on the C# image.
• Buffering proxies and antivirus: a proxy or antivirus scanner that holds a whole response before passing it on could keep the client from seeing data within 30 s. On a large bundle that could retry forever. A per-request size cap would prevent this; it is left out for now.
• Retries: failures and timeouts still retry immediately, with no backoff and no limit, as before. Each retry now carries on from the bytes already received.
• Cancel during a stall: a stalled read notices Cancel only when the 30 s idle timeout fires.

Test plan

• LibChorus.Tests HgResume tests pass on net8.0 (71/71 before stacking on Update SIL.*Tasks to 4.0.0 and fix restore against libpalaso 18.0.0 #404; the key 23 re-run after)
• net462 run in CI
• LibChorus.IntegrationTests (large clone, connection cut four times) pass in CI against the hgresume image
• Re-run the integration tests against a server with Send bundle chunks from the file, and test against several Chorus clients hgresume#31

🤖 Generated with Claude Code

---

This change is 

[github-actions]

Test Results

       9 files  ±  0     353 suites  +6   52m 2s ⏱️ - 3m 34s
1 062 tests +20  1 005 ✔️ +19    56 💤 ±0  1 ❌ +1 
3 345 runs  +56  3 221 ✔️ +55  123 💤 ±0  1 ❌ +1 

For more details on these failures, see this check.

Results for commit b755902. ± Comparison against base commit 050c057.

This pull request removes 1 and adds 21 tests. Note that renamed tests count towards both.

Pull_BundleInMultipleChunks_Success

CalculateEstimatedTimeRemaining_FromRate_ExpectedString(0,1000.0d,"(less than 1 minute)")
CalculateEstimatedTimeRemaining_FromRate_ExpectedString(1000,0.0d,"")
CalculateEstimatedTimeRemaining_FromRate_ExpectedString(300000,1000.0d,"(about 5 minutes)")
CalculateEstimatedTimeRemaining_FromRate_ExpectedString(59000,1000.0d,"(less than 1 minute)")
CalculateEstimatedTimeRemaining_FromRate_ExpectedString(60000,1000.0d,"(about 1 minute)")
ChunkSizeToAskFor_CaughtUp_TheRest
ChunkSizeToAskFor_HaveItAll_TheRest
ChunkSizeToAskFor_NothingKnownButDataOnDisk_OneByteProbe
ChunkSizeToAskFor_NothingKnown_OneByteProbe
ChunkSizeToAskFor_ServerHasMoreThanOffset_AllButTheLastByte
…

♻️ This comment has been updated with latest results.