[emmy-devop]

Requested by @drafty

🔴 Review tier: HIGH — human review required

• Release: merging to main publishes a new asknews release to PyPI.
• Contract: new public kwarg stream_sources_inline on get_deep_news (sync + async), new stream chunk type in both stream unions, two new optional Article fields.
• Depends on an unmerged upstream change: asknews/api !450 (chore/src-batch-spec), the spec this was written against (0.32.8 + the batch chunk; same contract as emergentmethods/asknews-sdk#7). Merge after !450 is promoted to dev.

What

Types the deep news inline source-batch streaming feature (engine v2.0), mirroring chat-service dto/v1/deepnews.py @ 4fe4a794 / api !450:

• CreateDeepNewsRequest.stream_sources_inline: Optional[bool] = False, and the matching stream_sources_inline: bool = False kwarg on ChatAPI.get_deep_news / AsyncChatAPI.get_deep_news (all overloads; sent in the request body). Engine v2.0 only; off by default.
• CreateDeepNewsResponseStreamSourceBatch (object: "chat.completion.source.batch"; id, created, turn_index, call_index, calls_in_turn, tool_call_id, tool_name, kind: DeepNewsToolCallKind, status: DeepNewsToolCallStatus, source_ids, sources) with the kind-discriminated entry union CreateDeepNewsResponseStreamSourceBatchEntry = …NewsEntry (SearchResponseDictItem) | …WebEntry (WebSearchResult) | …GraphEntry (AnyUrl) | …ChartEntry (ChartResponse), each kind + source_id + source + optional citation_number. Sources already streamed are referenced by id only; the final chat.completion.source chunks are unchanged.
• The chunk is added to both streaming unions, CreateDeepNewsResponseStream and CreateDeepNewsResponseStreamV2 (tag chat.completion.source.batch). Today, main raises ValidationError ("Input tag 'chat.completion.source.batch' … does not match any of the expected tags") as soon as a v2.0 stream with stream_sources_inline=true yields a batch chunk — the untyped gateway already proxies it.
• Article.centroid_distance / Article.cluster_probability: Optional[float] = None (the spec makes them nullable and no longer required; the SDK model did not carry them at all, so this is additive).

No README change (its example is the non-streaming v1.5 call); no version bump, no CHANGELOG.md edit (commitizen).

How I tested

• uv sync --frozen, then uv run coverage run -m pytest -q → 183 passed (169 on main + 14 new in tests/api/test_chat_deepnews_source_batch.py), coverage 93 %.
• ruff check asknews_sdk tests/api/test_chat_deepnews_source_batch.py clean; ruff format --check clean on the changed files (ruff 0.16.0 — uvx could not create its tool dir in the pod, so the pinned binary was used).
• New tests replay the gateway's recorded engine-v2.0 fixtures (tests/fixtures/research_canvas/*.jsonl, copied verbatim from api !450 ← chat-service 4fe4a794, 44 chunks / 6 batch chunks) through respx as SSE: sync and async get_deep_news(stream=True, engine="v2.0", stream_sources_inline=True) yield typed CreateDeepNewsResponseStreamSourceBatch instances alongside the existing …StreamChunkV2 / …StreamSource chunks; request body carries stream_sources_inline (and defaults to false); recorded serial batches (call-0/call-1, news:2 re-found → in source_ids, not re-sent) and mixed-kind batches (web + news sharing a URL → web:1 / news:2; search_wikipedia → kind: "wiki" with web entries) are typed per entry kind; unknown entry kinds are rejected; both unions accept the discriminator; Article validates with the cluster metrics absent, null, or set.
• Verified the pre-change failure mode on main (the ValidationError quoted above) against the same fixture.

Risk / rollback

Additive: existing streams without the flag parse exactly as before; a stream with stream_sources_inline=true goes from crashing to yielding typed chunks. Callers iterating a v2.0 stream should now expect a third chunk type (isinstance(token, CreateDeepNewsResponseStreamSourceBatch)). Merging publishes to PyPI (keep the fix: subject so commitizen cuts a release); revert = revert this one commit, then another release.

Docs

Second commit (docs:), on top of main's new cookbook (the branch already contained d8b96e4 chore: Add a cookbook, so no merge/rebase was needed):

• cookbook/recipes/12_deep_research_basics.py (+ regenerated cookbook/markdown/12_deep_research_basics.md via python cookbook/build.py): stream_sources_inline row in the parameter table and a new "Streaming sources per tool call (stream_sources_inline)" section — what it does and why (sources per tool call while research runs, for progressive UIs / citations as they arrive), the requirements (stream=True, engine="v2.0"), a field table for CreateDeepNewsResponseStreamSourceBatch, a runnable sync example that handles the batch chunk, switches on the entry kind (news/web/graph/chart), dedupes by source_id (re-found sources come back in source_ids only), resolves each call's source_ids against what was streamed, and maps citation_numbers; a short async variant; and the note that the final chat.completion.source messages are still emitted.
• cookbook/README.md: recipe 12 row mentions the new section.
• Verified: the example's code cell was executed as-is against the recorded engine-v2.0 fixtures (tests/fixtures/research_canvas/*.jsonl, both files) through the branch's models — 6 batches, 8 unique sources, re-found news:2/a2 deduped. pytest -q 183 passed; ruff check asknews_sdk clean (the cookbook is outside task lint, and its existing recipes share the same long-table-line style).

Follow-up (not in this PR)

asknews/frontend sends stream_sources_inline as an untyped extra key (apps/asknews/src/lib/chat/research-canvas/replay-request.ts:55-62) and parses batch events untyped (v2-adapter.ts:337-340); docs/features/research-canvas/RESEARCH_CANVAS.md:540-542 names this as the gap. The TypeScript side is emergentmethods/asknews-sdk#7.

CI fix (4740397): cookbook/ is excluded from ruff in pyproject.toml, and the lint task now resolves its target via a Taskfile var instead of a shell export that did not survive between cmds entries (which made CI lint the whole repo). task lint → 0 errors; 183 tests pass.