Summary
Intermittent 403 {"message":"Authentication failed. Please check your credentials.","type":"permission_error"} rejections occur on the commandcode/meta/muse-spark-1.3-contributor model path via the Provider API.
This issue is characterized by 79 occurrences across 5 independent sessions (tracked between 2026-09-26 and 2026-09-28, with a spike of 68 on 2026-09-28). Every failure reports empty content and zero token usage (input: 0 / output: 0), indicating the request is rejected at the authentication layer before generation kicks off. Re-logging does not resolve the issue, but the identical prompt succeeds immediately on manual retry, confirming the stored key is valid.
Implemented local agent_before_settle loop extension (retrying up to 3 times) still hits consecutive failures, ruling out simple single-packet drops.
Expected Behavior
Transient authentication failures should be retried internally by the client or surfaced with a trackable server-side request_id. A valid API key should not intermittently flap to a 403 status on ~1% of turns when zero usage has occurred.
Actual Behavior
The current turn dies instantly with the following payload footprint:
- stopReason: "error"
- content: []
- usage.totalTokens: 0
- errorMessage: 403: {"message":"Authentication failed. Please check your credentials.","type":"permission_error"}
The subsequent identical request succeeds immediately. No server request ID is returned to correlate logs on the infrastructure side.
Steps to reproduce the issue
not sure if this only occurs only on pi
- use pi/other harness
- use meta/muse-spark-1.3-contributor
- let it run for few minutes
- the 403 error will pop up and stop the agents/run
Command Code Version
pi 0.87.1 | pi-commandcode-provider 0.7.2
Operating System
Linux
Terminal/IDE
ptyxis
Shell
fish
Session file (optional)
No response
Fix prompt (optional)
No response
Additional context
No response
Summary
Intermittent 403 {"message":"Authentication failed. Please check your credentials.","type":"permission_error"} rejections occur on the commandcode/meta/muse-spark-1.3-contributor model path via the Provider API.
This issue is characterized by 79 occurrences across 5 independent sessions (tracked between 2026-09-26 and 2026-09-28, with a spike of 68 on 2026-09-28). Every failure reports empty content and zero token usage (input: 0 / output: 0), indicating the request is rejected at the authentication layer before generation kicks off. Re-logging does not resolve the issue, but the identical prompt succeeds immediately on manual retry, confirming the stored key is valid.
Implemented local agent_before_settle loop extension (retrying up to 3 times) still hits consecutive failures, ruling out simple single-packet drops.
Expected Behavior
Transient authentication failures should be retried internally by the client or surfaced with a trackable server-side request_id. A valid API key should not intermittently flap to a 403 status on ~1% of turns when zero usage has occurred.
Actual Behavior
The current turn dies instantly with the following payload footprint:
The subsequent identical request succeeds immediately. No server request ID is returned to correlate logs on the infrastructure side.
Steps to reproduce the issue
not sure if this only occurs only on pi
Command Code Version
pi 0.87.1 | pi-commandcode-provider 0.7.2
Operating System
Linux
Terminal/IDE
ptyxis
Shell
fish
Session file (optional)
No response
Fix prompt (optional)
No response
Additional context
No response