Skip to content

fix(backtests): keep a pipeline order's zero confidence instead of reading it as 0.75 - #673

Open
subidhkhanal wants to merge 1 commit into
Open-Finance-Lab:mainfrom
subidhkhanal:fix/pipeline-zero-confidence
Open

subidhkhanal wants to merge 1 commit into
Open-Finance-Lab:mainfrom
subidhkhanal:fix/pipeline-zero-confidence

Conversation

@subidhkhanal

Copy link
Copy Markdown
Contributor

The bug

pipeline_output_to_decision (infrastructure/llm/pipeline_runner.py, orders branch) read an order's confidence as:

"confidence": float(order.get("confidence", 0.75) or 0.75),

The or 0.75 was meant as a default for a missing confidence. In Python, 0 or 0.75 is 0.75, though, because 0 is falsy. So a pipeline order with confidence: 0 ("not confident at all") was read as 0.75, passed the 0.3 floor in make_trading_decision_with_llm and traded.

Reproduced on main before the fix. Running a pipeline reply of {"orders": [{"symbol": "AAPL", "side": "buy", "qty": 10, "confidence": 0}]} through PortfolioManager.make_trading_decision_with_llm placed the buy:

✅ BUY AAPL: 10 shares @ $100.00 (conf: 75%)

The fix

Only a missing (None) or empty ("") confidence takes the 0.75 default; a real 0 is kept:

"confidence": float(
    0.75
    if order.get("confidence") in (None, "")
    else order["confidence"]
),

Missing and empty confidence behave exactly as before. Non-numeric confidence (e.g. "high") is also unchanged: it still raises, which is #663's scope and is left for that issue.

Tests

  • test_pipeline_runner.py::test_pipeline_order_with_zero_confidence_keeps_it: confidence 0 stays 0.0, and a missing confidence is still 0.75.
  • test_portfolio_manager_move.py::test_pipeline_order_with_zero_confidence_does_not_trade: end to end, a zero-confidence pipeline order produces no action.
  • Both failed on main (assert 0.75 == 0.0, and the buy above) and pass with the fix.
  • Focused suites (tests/infrastructure/llm, tests/backtesting, tests/domain/backtesting, tests/conformance): 1133 passed, 0 failed.

Related: #663 quotes the same line, for non-numeric values raising. This PR does not close it.

…ading it as 0.75

pipeline_output_to_decision read an order's confidence as float(value or 0.75), so confidence 0 became 0.75, passed the 0.3 floor and traded. Only a missing or empty confidence now takes the 0.75 default.
@vercel

vercel Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

@subidhkhanal is attempting to deploy a commit to the allan-feng's projects Team on Vercel.

A member of the Team first needs to authorize it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant