[Yaomeng1749]

When the notifier is waiting for notifications, Listen (for a topic's first subscription) and unlisten (for its last) hand their LISTEN or UNLISTEN to the run loop through sendInterruptAndReceiveResult, which gives up after 5 seconds with timed out waiting for interrupt operation. The channel the result comes back on was unbuffered, so when the operation finished after the caller had given up, the run loop blocked forever sending a result that nothing would receive. The notifier never waited for notifications again, and its Stop never returned.

All it takes is for the run loop to need more than 5 seconds to get to the operation and finish it, for example on a slow or unresponsive connection, where LISTEN and UNLISTEN each get the 10-second listenerTimeout. These calls come from producers and the elector as a client starts and stops, so once the notifier was stuck:

• Client.Stop never completed, since stopping the client stops the notifier.
• A failed Client.Start also stops its services before returning, so when a producer's or the elector's Listen was the one that timed out, Client.Start never returned.
• If the client kept running, which is only possible after removing its last queue with Queues().Remove and adding one back, its producers also stopped hearing about job cancellations and queue changes, since with a listener they don't poll for them.

I think this is what the logs in #644 show. The producer's river_insert subscription is removed exactly 5 seconds after it's added, the notifier logs Listening on topic for it 4 ms later, the elector's unlisten then times out as well, and the log ends without the notifier's Run loop stopped. The shutdown there looks like Client.Start cleaning up after the producer's failed Listen rather than a regular stop: there's no River client started, no Client: Stopping producers, and the producer never logs Run loop started. That would explain why Start never returned. With this change, it would return the timeout error instead.

The fix gives the channel a buffer of one, as #1205 did for leaderResignedChan, so the run loop's send always goes through, and the result is dropped if the caller has given up. A longer timeout would only make this less likely, since it also has to cover the time before the run loop gets to the operation. I didn't add logging because the caller already returns or logs the timeout error. This also doesn't change what a timeout leaves behind. For example, a timed-out unlisten still returns before removing its subscription, as before.

TestNotifier_Start/InterruptTimeout uses synctest and a listener whose Listen takes 6 seconds. It checks that Listen returns the timeout error, and that once the operation finishes, the run loop goes back to waiting for notifications, runs another Listen through an interrupt, and stops. Without the fix, the run loop is still stuck sending the first result, so the test fails and synctest reports the deadlock.

I checked the Java, JS, Ruby, and Rust ports, and they aren't affected. They LISTEN on a fixed set of topics when they connect, so none of them hands subscription changes to a running wait loop.

Locally against Postgres 16, the new subtest passes with -race -count=20 on Go 1.26 and 1.27, and make test, make test/race TEST_DATABASE=postgres, and make lint pass.

#1361 also adds a CHANGELOG.md entry under Unreleased, so whichever of the two lands second will need a rebase there, which I'm happy to do. It changes notifier.go and its tests too, but not sendInterruptAndReceiveResult, and apart from CHANGELOG.md it merges cleanly with this.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-10-29T16:51:26.925165Z We couldn’t complete this review. Comment @codex review to try again.	e1476b8	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[brandur]

Thanks!