Skip to content

rust mode: historian reattach after a module restart waits the producer's 600 s default instead of the configured historian_timeout_ms (residual of #507) #522

Description

@iceteaSA

Refs are against master 62bf58c0c. #507's fix (5641bfc) made the normal historian attempt honour historian_timeout_ms. One path still doesn't: reattach after the module restarts mid-run.

Path

When the module comes back and finds a durable historian phase of AwaitingProducer, it rebuilds a producer with factory.connect(&project_root) (crates/mc-module/src/lib.rs:5525-5553) and awaits the existing run. HistorianReattachRequest (crates/mc-module/src/historian.rs:1254-1271) has no timeout field. The await goes through await_output (crates/mc-module/src/historian_producer.rs:838-851):

pub async fn await_output(&mut self, run_id: &str) -> Result<ProducerOutput, HistorianProducerError> {
    self.await_output_with_timeout(run_id, self.config.await_timeout).await
}

self.config.await_timeout is the producer default (600 s), not the per-request historian_timeout_ms the normal attempt now uses.

Effect

With historian_timeout_ms set below 600,000 (the schema allows 60,000 upward), a restart during a run makes the reattach wait up to ten minutes before it abandons or refires, instead of the configured budget. A per-session latch (lib.rs:5436-5445) prevents duplicate reattach tasks, so the only cost is the longer wait. With the default timeout the two values coincide and nothing changes. On my host no project sets the key, so I haven't reproduced this; it's a source trace.

Suggested fix

Carry the effective historian_timeout_ms into HistorianReattachRequest, or persist it with the durable phase, and call await_output_with_timeout with it on reattach.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions