Skip to content

fix(broadcast): keep a caught-up consumer group on a quiet stream (2.1) - #292

Closed
cosmin-staicu wants to merge 1 commit into
release/2.1from
fix/stream-caught-up-group-2.1
Closed

cosmin-staicu wants to merge 1 commit into
release/2.1from
fix/stream-caught-up-group-2.1

Conversation

@cosmin-staicu

Copy link
Copy Markdown
Member

Backport of #290 to 2.1.x.

The stream maintainer counted a consumer group with consumers as stale once its last-delivered-id was older than MaintainerTrimInterval. It never checked whether anything newer had been written. On a stream with no traffic, every live group was quarantined, deleted after MaintainerQuarantineInterval, and recreated by its reader on the next NOGROUP. That repeated every cycle, with warnings, LogRecreatingTopic, and XGROUP calls that sometimes timed out.

A group whose last-delivered-id has reached the stream's last-generated-id is now healthy, and a quarantine it was under is cleared.

  • Both parts of the ID are compared, so a group behind by one entry in the same millisecond still counts as behind.
  • If either ID can't be parsed, the old rule applies.
  • A caught-up group whose consumer is dead is no longer removed on a quiet stream. It is quarantined and deleted as before once the stream gets a new entry.

Tests:

  • Unit tests: a caught-up group is not quarantined; a caught-up group under an elapsed quarantine is released, not deleted; a group behind in the same millisecond is still quarantined; a stream head that can't be read keeps the old rule.
  • Integration test against Redis: on a quiet stream, a caught-up group is kept and a group that is behind is deleted.
  • The tests fail with the old code, and with a check that compares only milliseconds.
  • The full suite passes on net8.0 and net10.0 with the Redis integration tests on. The Release -warnaserror build is clean.

The code is the same as #290. Only the CHANGELOG entry differs, because it goes into 2.1's own Unreleased section.

The maintainer counted a group with consumers as stale once its last
delivery was older than MaintainerTrimInterval, without checking whether
anything newer had been written. On a stream with no traffic every live
group was quarantined, deleted after MaintainerQuarantineInterval and
recreated by its reader on NOGROUP. A group whose last-delivered-id has
reached the stream's last-generated-id is now healthy.

Signed-off-by: Cosmin Staicu <cosmin.staicu@uipath.com>
@sonarqubecloud

Copy link
Copy Markdown

@cosmin-staicu

Copy link
Copy Markdown
Member Author

Folded into #293.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant