Backend service restart can race listener-name release #143

Activity

1. unilynx commented on Oct 27, 2026

   

   unilynx

   on Oct 27, 2026

   Member

   More actions

   Repeatedly restart the existing platform:calljs

   how? wh service restart platform:calljs or the underlying nodeservices?
2. RichardHoekstra commented on Oct 27, 2026

   

   RichardHoekstra

   on Oct 27, 2026

   ContributorAuthor

   More actions

   Via the platform:services.node manager RPC, from native HareScript. The nodeservices process stays running. After a warm-up CallJS, the async macro does:

   OBJECT manager := AWAIT OpenWebHareService("platform:services.node");
   FOR (INTEGER iteration := 0; iteration < 10; iteration := iteration + 1)
     AWAIT manager->Restart("platform:calljs");
   

   The failure occurs in NodeServicesClient.restart → #startService → runBackendService → IPCPortImpl.activate, when rebinding webhareservice:platform:calljs. I didn’t invoke wh service restart platform:calljs in this repro.
3. unilynx commented on Oct 27, 2026

   

   unilynx

   on Oct 27, 2026

   Last edited by unilynx

   Member

   More actions

   awaits unregister acknowledgement and socket cleanup, and serializes service transitions

   Hmm. But this increases downtime (service unreachability) and also doesn't help to solve an existing problems: incoming connections might still be on their way to the calljs process now shutiting down.

   I was working on a different approach but not sure when I can continue on it - moving backend service connections (such as calljs) to unix sockets. Then we can do a replace of the existing socket (perhaps even atomic, possibly using symlinks) and have the old service instance handle existing connections and the new instance listen for new connections. And as a bonus it doesn't require whmanager to forward ipc messages.

   There is some basic code that implements TS->TS unix connections, but we're missing at least, compared to IPC:

   • advanced type support, eg blob references.
   • HS client -> TS service connections. (not caring for HS services offering unix sockets)
4. RichardHoekstra commented on Oct 27, 2026

   

   RichardHoekstra

   on Oct 27, 2026

   Last edited by RichardHoekstra

   ContributorAuthor

   More actions

   I traced the existing Unix-socket implementation. addUnixSocket() already starts a temporary listener and renames it onto the published service path.

   The two IPC-parity gaps you mentioned are visible in the current paths:

   • The socket transport uses typed JSON. IPC has explicit blob marshalling, including disk-backed references. Equivalent representation and lifetime/ownership rules still need covering.
   • Native OpenWebHareService() still connects through global IPC. HS clients need a compatible socket/protocol path to TS services; HS-hosted services can remain outside that scope.

   I also runtime-tested one lifecycle detail using Unix-only services on the unmodified official 6.2 image for main a91a7a2:

   1. Start A and connect a client.
   2. Start B under the same name: a fresh client reaches B while the existing client still reaches A.
   3. Close A: both established connections keep working, but the published socket path disappears and fresh connections fail.

   Reproduced in all three runs. A control that explicitly waited for A’s cleanup before starting B passed.

   WebHareService.close() unconditionally removes the shared service path, so overlapping generations need cleanup that preserves the replacement’s endpoint.

   That probe covers basic TS→TS overlap only; it doesn't validate blob transfer or HS clients.
5. RichardHoekstra commented on Oct 27, 2026

   

   RichardHoekstra

   on Oct 27, 2026

   ContributorAuthor

   More actions

   Follow-up after testing the HS side on the same unmodified 6.2 image:

   After warming up the service through regular CallJS, a small native HS client using the existing ConnectSocket() could send constructor and invoke messages directly to the stock platform:calljs Unix listener. A synthetic 20 + 22 function returned 42 on all three invocations.

   So native HS socket transport and basic CallJS dispatch already work. The remaining HS work includes integrating that protocol into the service client and preserving the existing synchronous CallJS behavior.

   The type-parity probe also gave concrete examples: ordinary HS typed JSON preserves Money and Date, but encodes blobs as base64 strings and sends INTEGER64 values as JSON numbers, losing precision above the safe integer range. In the other direction, the current HS typed decoder rejects the lowercase "bigint" tag emitted by TS. The existing HS→IPC route preserved those values.

   That gives us a working transport baseline and specific compatibility cases for the value codec.
6. RichardHoekstra commented on Oct 27, 2026

   

   RichardHoekstra

   on Oct 27, 2026

   ContributorAuthor

   More actions

   Runnable fixtures: source and per-case notes, with a disposable Docker runner.

   From that checkout:

   python3 whtree/modules/platform/scripts/repro/run.py --docker 143
   python3 whtree/modules/platform/scripts/repro/run.py --docker 143-handover
   python3 whtree/modules/platform/scripts/repro/run.py --docker 143-codec
   python3 whtree/modules/platform/scripts/repro/run.py --docker 143-calljs

   These cover the original manager-RPC restart failure and the three follow-up probes: Unix generation cleanup, HS value-codec compatibility, and basic native HS access to the stock CallJS socket. Each runs in a fresh instance of the pinned official 6.2 image for a91a7a28.

   The notes include the observed results and controls. No proposed restart or transport implementation is installed.