Repository navigation
Default report frequencies and reject values below 100ms - #466
Merged
Merged
Conversation
With a progress or state callback configured and the frequency left at 0 (or set negative), the reporting loop waits on time.After(0) and posts to the callback as fast as it can.
ProgressReportFrequency and StateReportFrequency now default to 60000ms like other optional settings, and ValidateConfig rejects values below 100ms when the matching callback is configured. Fixes #214.
shivnagarajan
approved these changes
Oct 22, 2026
shivnagarajan
left a comment
Contributor
There was a problem hiding this comment.
Just a few comments. Nothing super critical, but just some minor adjustments suggested.
| if c.ProgressReportFrequency == 0 { | ||
| c.ProgressReportFrequency = defaultReportFrequencyMs | ||
| } | ||
| if c.ProgressCallback.URI != "" && c.ProgressReportFrequency < minReportFrequencyMs { |
Contributor
There was a problem hiding this comment.
it is a bit strange that 0 is handled differently from values < minReportFrequencyMs (100)
Contributor
Author
There was a problem hiding this comment.
Kept 0 as “unset/use the 60-second default,” consistent with optional Go integer fields so it wouldn't break existing setups, but apply new default
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #214. Supersedes #215.
Problem
ProgressReportFrequencyandStateReportFrequencypreviously had no defaults or validation. With a callback URI configured, zero or negative intervals madetime.Afterfire immediately, repeatedly serializing and posting progress/state. The original copydb smoke observed 1,053 progress POSTs in 1.5 seconds when the frequency was omitted.Policy
For each reporting frequency:
Zero is an unset/default sentinel, not a way to disable reporting. This follows the convention used by other optional numeric configuration fields. Explicit invalid nonzero values are rejected rather than silently clamped.
Library-path protection
The policy lives in one private
Config.validateReportFrequencieshelper, called by bothConfig.ValidateConfigandFerry.Initialize.Initializeinvokes it after establishing the default error handler but before opening database connections. This protects the normal libraryInitialize→Start→Runlifecycle even when callers do not invokeConfig.ValidateConfig, without duplicating policy in the reporting loops or revalidating unrelated settings insideRun.Review updates
The library-path regressions failed before this correction: initialization retained zero frequencies and accepted 99 ms. They pass with the shared helper.
Verification
go test -count=1 -p=1 . ./test/go ./copydb/test ./sharding/test: all four suites passed.test/integration/callbacks_test.rb: four tests, zero failures.Config.ValidateConfig.Initializeset both intervals to 60000 ms; the run copied three rows and emitted two progress POSTs and no state POSTs instead of flooding either callback.Updated the branch with current
mainand retained all independent changelog entries when resolving the merge conflict.CI follow-up: interrupt/resume shutdown
The failed Go 8.0/GTID job stopped before testing because pulling the MySQL image returned HTTP 502. The two Ruby 8.4 jobs instead timed out in the interrupt/resume suite.
Two test callbacks paused the data iterator with an unbounded
sleep. After the Go subprocess exited, WEBrick waited forever for those Ruby request handlers, leaving the test harness stuck inkillwhile joining its server thread. A focused real-process/HTTP probe reproduced that exact shutdown state.The callbacks now wait for the subprocess thread to finish, so they release when the child exits. A queue barrier ensures both affected regressions actually enter the paused-copy path before sending TERM; no data/verification assertions were removed, and no retries or longer job timeouts were added.
Post-fix verification used isolated Percona 8.4.4 servers with the CI server flags:
Both were single-process
bundle exec rake testruns. The two forced-pause regressions also passed individually in both modes. The standalone shutdown probe changed fromchild_alive=false, shutdown_finished=falsetochild_alive=false, shutdown_finished=true.Temporary database-port overrides used for isolated verification were restored before committing.