[ZhulongNT]

Fixes #837.

Follow-up to the closed issue #804 (the original "DSML markers leak into assistant content" report) and the closed, unmerged PR #805 (the first per-chunk content filter), and complementary to merged workaround f7e4cc5 and the still-open PR #836. Those cover the symptom end (exact trailer retry / 200-char tail heuristic); this PR fixes the marker matching itself, which is what makes </||DSML|| parameter> survive all of them.

What this fixes

Three holes, all in llmcore.py, plus a guard that is no longer tied to one hardcoded suffix in ga.py:

1. Closing tags and pipe variants were never matched. The marker regex required < to be immediately followed by the pipe+DSML sequence (so </...> closing tags escaped) and only accepted a single ｜ (U+FF5C), not |, ｜｜ or ‖. Replaced with one variant-agnostic marker pattern (optional /, [|｜‖]{1,2}, optional whitespace around the DSML token).

2. Markers split across stream chunks were never seen. Filtering ran per chunk behind a "DSML" not in text fast path, so "hello<|" + "|DSML||tool_calls>world" matched nothing. Added a small stateful filter (_DsmlStreamFilter) that holds a trailing partial <... back and re-joins it with the next chunk, so a split marker is matched exactly once and in order; the fast path remains for the common case.

3. Non-streaming responses were never filtered (_parse_openai_json), and neither were the text blocks of a streamed response in some paths.

4. ga.do_no_tool now retries when llmcore latches a leak (response.dsml_leak) - a positive signal, so a leak anywhere in the turn (not just at the very end) and any marker family triggers a clean regeneration - while the cheap tail check stays as a fallback for responses produced elsewhere.

Tests

frontends/tests/test_dsml_marker_strip.py (11 tests, runs in the existing desktop CI job, no network/API key): marker spelling variants, closing tags of invoke/parameter/tool_calls families, cross-chunk splits (the exact </||DSML|| parameter> case), the non-streaming path, the latch contract ga.py depends on, and non-regression cases (ordinary text untouched; the payload between two leaked markers is preserved).

Risk

Only text content is touched, and only when a DSML-shaped marker is actually present; real tool calls are parsed by the provider adapter before this path. Leak-free text takes the same fast path as before (a single "DSML" in text check). The filter holds at most a trailing <... fragment back between chunks, and flushes at stream end, so no content can be lost.

---

Update 2026-10-27 — rebased + two more real-world variants

Rebased onto main@e4d956f (30 commits, incl. a9a36b4), so the PR applies
cleanly again; the one conflict in ga.py was resolved by keeping this PR's
text-channel recovery block and dropping the now-deleted plan_mode guard.

Two new commits, both driven by sessions that still stalled after the
above fixes:

1. llmcore: recover more text-channel call variants (#837) — a call emitted as
   <code_run><script>...</script><parameter name="type">python</parameter></code_run></invoke></function>
   was rejected because (a) <script> was not accepted as a parameter container,
   (b) anything after the closing tag made it "not a call", (c) a call truncated
   mid-value was not flagged as partial, so no retry happened. Now: container
   names are parsed and mapped to declared parameter names, a tail of stray
   closing tags is tolerated, and a value-less/truncated call returns partial
   so the turn is retried instead of silently ending.
2. ga: don't treat quoted DSML markers as a leak — the guard matched the marker
   anywhere in the last 200 chars, so a reply that quotes it (writing docs,
   answering "what is this bug", pasting the upstream regex) was rejected as a
   leaked tool call. Code spans/fenced blocks that are followed by prose are now
   stripped before matching; a fence at the very end still counts, because that
   is exactly where a leaked call hides.

Verification: the two dead sessions from 2026-10-27 (model_responses_353300.txt,
17:08 and 17:59 — both followed by the user typing "continue") now parse into a
code_run call, with 6 new tests added (frontends/tests/ 30 passed).
Full frontends/tests/ on this branch: 189 passed, 1 pre-existing failure
(test_conductor_model_routing, also fails on main@e4d956f).

[ZhulongNT]

Pushed 958e2d5 (4th commit): the text-channel rescue + stream-observability half of this fix.

What it adds on top of the DSML stripping:

Execute instead of ignoring — llmcore.split_text_tool_calls(text, known_tools) recognises a call the model wrote as literal markup and GA now runs it:

• complete <code_run><parameter name="script">…</parameter></code_run> → real call (also handles the attribute form <code_run script="">…</code_run> and raw <code_run>…print(1)</code_run>);
• <tool_use>{"name":…,"arguments":…}</tool_use> JSON form;
• a marker in the middle of prose, or text after the closing tag, is not treated as a call, so prose that merely mentions <code_run> is safe.

Retry instead of stopping — a truncated marker at the end of the turn (<code_run">, <tool_use>{"name": …) means the model was about to call a tool and got cut off; do_no_tool now asks it to regenerate the turn once instead of silently ending (this is exactly the 2026-10-17 session where the turn ended on <code_run">).

Stream observability (responses api_mode) — response.completed now also parses the final output snapshot, output_item.done is handled (previously only .added), a stream with no terminal event is flagged ([!!! stream ended without a terminal event]), and response.incomplete warns even when some text already arrived. These turned a torn turn into a normal-looking answer before.

Tests: frontends/tests/test_text_channel_tool_call.py (9 cases) + adjusted the dsml_f.feed count in the existing DSML test → frontends/tests 293 passed, 3 failures are pre-existing Windows symlink/path issues unrelated to this change.

Note: GitHub did not create a check suite for 958e2d5 (the only run on this PR is for c78f0f1); a re-run would need to be triggered from the Actions tab.