Skip to content

4.6.0+ with zod ≥ 4.4: every run completion waits a fixed 6 s (cancels 31 s) because FLUSH/CANCEL IPC acks are rejected #4979

Description

@marcus-dk

What happens

Since 4.6.0, in projects that resolve zod to 4.4 or newer, every run on a managed worker takes ~6.1 s longer between the task finishing and the completion reaching the engine. Compute (usage_duration) is unchanged; execution_duration - usage_duration goes from ~0.1–0.25 s on 4.5.15 to 6.10–6.29 s on 4.6.0 and 4.6.2, for every task, warm or cold. Cancellation is hit harder: the CANCEL ack budget is 30 s + 1 s. Still present on main (c2b7a72) and in 4.6.4: zodIpc.ts is unchanged since #4039.

We see it on every task we run: the gap appears exactly at the SDK upgrade and never drops below ~6.09 s, where it was ~0.1 s before.

Why

TaskRunProcess.cleanup() sends FLUSH to the task run process with sendWithAck(..., timeoutInMs + 1_000) (5 s + 1 s) before completeRunAttempt. FLUSH and CANCEL declare callback: z.void(), so the child acks with { type: "ACK", id, message: undefined }. process.send JSON-serialises the packet and drops the message key. In zodIpc.ts the Packet schema declares message: z.any(); under zod 3 a z.any() key was optional at parse time, under zod 4.4+ it is required (see the version table below). Since #4039, zodIpc.ts imports zod/v4, and @trigger.dev/core usually resolves zod from the user's project (^3.25.56 || ^4.0.0). So any project that resolves zod 4.4 or newer (this repo pins 4.5.4) fails Packet.safeParse on every ack, #handlePacket returns silently, and sendWithAck waits out its full timeout. Nothing is logged because the rejection is swallowed by tryCatch.

Repro (zod alone)

zod 4.6.5: z.discriminatedUnion("type", [...ACK { message: z.any(), id }]).safeParse({ type: "ACK", id: 0 })
  -> message: Invalid input: expected nonoptional, received undefined

The same schema across versions (root zod and zod/v4 behave the same):

zod 3.25.76  accepted
zod 4.3.6    accepted
zod 4.4.0    REJECTED
zod 4.5.4    REJECTED

That is probably why this has gone unreported: projects on zod 3.x or 4.0–4.3 are unaffected, and billed compute (usage_duration) does not change.

Repro (two ZodIpcConnections over a forked child, 3 s ack timeouts)

core 4.6.2:  FLUSH: REJECTED after 3002ms (sendWithAck() timeout)   CANCEL: REJECTED after 3001ms
core 4.5.15: FLUSH: ack in 2ms                                       CANCEL: ack in 0ms

End-to-end confirmation (trigger dev, same server, same one-task project, six runs each)

worker task compute gap between task end and completion
stock trigger.dev@4.6.2 14 ms 6.27 – 6.34 s
same install, only message: z.any().optional() on the ACK packet in core's dist 14 ms 0.40 – 0.46 s

The dev controller runs the same TaskRunProcess.cleanup() → FLUSH → completeRunAttempt sequence as the managed controller, and that line is the only change between the two workers.

Fix

Make the ACK packet's message optional again (z.any().optional()), which restores the zod 3 type and runtime behaviour. Only the ACK packet can legitimately carry undefined (the two void callbacks); no message catalog entry has an undefined payload, so the messageSchema.payload: z.unknown() envelope is unaffected. Happy to open a PR with this change and a JSON-round-trip test that fails on main and passes with the fix.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions