Refs are against master 62bf58c0c. #507's fix (5641bfc) made the normal historian attempt honour historian_timeout_ms. One path still doesn't: reattach after the module restarts mid-run.
Path
When the module comes back and finds a durable historian phase of AwaitingProducer, it rebuilds a producer with factory.connect(&project_root) (crates/mc-module/src/lib.rs:5525-5553) and awaits the existing run. HistorianReattachRequest (crates/mc-module/src/historian.rs:1254-1271) has no timeout field. The await goes through await_output (crates/mc-module/src/historian_producer.rs:838-851):
pub async fn await_output(&mut self, run_id: &str) -> Result<ProducerOutput, HistorianProducerError> {
self.await_output_with_timeout(run_id, self.config.await_timeout).await
}
self.config.await_timeout is the producer default (600 s), not the per-request historian_timeout_ms the normal attempt now uses.
Effect
With historian_timeout_ms set below 600,000 (the schema allows 60,000 upward), a restart during a run makes the reattach wait up to ten minutes before it abandons or refires, instead of the configured budget. A per-session latch (lib.rs:5436-5445) prevents duplicate reattach tasks, so the only cost is the longer wait. With the default timeout the two values coincide and nothing changes. On my host no project sets the key, so I haven't reproduced this; it's a source trace.
Suggested fix
Carry the effective historian_timeout_ms into HistorianReattachRequest, or persist it with the durable phase, and call await_output_with_timeout with it on reattach.
Refs are against master
62bf58c0c. #507's fix (5641bfc) made the normal historian attempt honourhistorian_timeout_ms. One path still doesn't: reattach after the module restarts mid-run.Path
When the module comes back and finds a durable historian phase of
AwaitingProducer, it rebuilds a producer withfactory.connect(&project_root)(crates/mc-module/src/lib.rs:5525-5553) and awaits the existing run.HistorianReattachRequest(crates/mc-module/src/historian.rs:1254-1271) has no timeout field. The await goes throughawait_output(crates/mc-module/src/historian_producer.rs:838-851):self.config.await_timeoutis the producer default (600 s), not the per-requesthistorian_timeout_msthe normal attempt now uses.Effect
With
historian_timeout_msset below 600,000 (the schema allows 60,000 upward), a restart during a run makes the reattach wait up to ten minutes before it abandons or refires, instead of the configured budget. A per-session latch (lib.rs:5436-5445) prevents duplicate reattach tasks, so the only cost is the longer wait. With the default timeout the two values coincide and nothing changes. On my host no project sets the key, so I haven't reproduced this; it's a source trace.Suggested fix
Carry the effective
historian_timeout_msintoHistorianReattachRequest, or persist it with the durable phase, and callawait_output_with_timeoutwith it on reattach.