Skip to content

perf(trajectory): stream capture parsing and source verification - #7757

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
shangzh0:codex/trajectory-capture-streaming-20261101
Oct 31, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
shangzh0:codex/trajectory-capture-streaming-20261101

Conversation

@shangzh0

@shangzh0 shangzh0 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

Long Codex captures were loaded as complete bytes, decoded text and split-line copies during usage registration. Each Research source verification also loaded whole artifacts. Stream JSONL parsing and SHA-256 verification while preserving the exact source digest, line provenance, counters and historical-source behavior.

Outcome basis: the concrete memory defect above; intended base is loopx-project/loopx:main at 04c5799b88f6c4cf12a224b7fa679f272e1e794e. The independently bounded memory regression and real counter projection below establish the delivered slice.

Author Declaration

Written by: model_agent — OpenAI Codex, GPT-6 family.

Implemented against docs/reference/trajectory-assessment.md at the base above, specifically source-verified registration and historical readback. The bounded-memory requirement is the concrete request in this PR.

Criterion Disposition Owner Decisive evidence
Preserve exact artifact digest and source lines implemented trajectory_assessment_store.py UTF-8/CRLF/no-final-newline and real counter projection
Recheck source contents, not metadata alone implemented _captured_view Same-size/same-time mutation becomes historical; deletion becomes unavailable
Avoid whole-file copies for selected path implemented _usage_rows, _source_digest 32 MiB fixture parses and verifies below a 4 MiB Python-allocation bound

Self-check: reviewed the full diff, source-verification contract and actual counter entrypoint; ran focused regression and adjacent registration tests. No scoring, solver or authority transition is changed.

Scope And Continuation

Complete within this scope. This changes parsing/verification allocation, not saved projections or UI. It does not require the separate tool-context inventory proposal. Optional native request payload loading and total end-to-end analysis cost remain outside this slice.

Validation

Tested revision: 99eb92496edd023c24817a1e023d5673360356c1. Run state: finished. Input classes: synthetic.

Check kind Result Evidence / limitation
unit / real_entrypoint passed 54 Python tests: streaming, native input capture and assessment registration/readback, including the actual typed counter projection
regression_parity passed Original-byte SHA and counter/source-line identity; malformed JSONL, missing source and equal-size/equal-time mutation
regression_parity passed Same 32 MiB synthetic parsing/hash workload: peak Python allocation 100,818,570 → 270,920 bytes, identical SHA-256. This is the selected path, not total analysis memory, token cost or end-to-end speed
static passed Ruff, semantic advisory, explicit public-boundary scan and git diff --check; the deliberately absent local Goal registry is a separate scan warning
integration not_run Full CI and installed application adoption were not qualified locally

The tests exercise production parsing, source verification and real typed readback. No raw private evidence is included.

Frontend / Visual Evidence

UI impact: none. Existing observer projection and rendering are unchanged. Before/after screenshots: N/A.

Type of Change

  • Bug fix
  • Refactoring (no functional changes to the saved observation)
  • Test update

LoopX Area

  • Capability or extension (benchmark-toolkit provider)
  • Benchmark boundary (read-only evidence observation)

Technical direction: existing S11 trajectory observer; no new capability or analysis owner. Shared-authority RFC fixture impact: N/A; no authority-provider or routing change.

Boundary Checklist

  • No private state, raw trajectories, credentials, local paths or internal links are published.
  • Scoped to capture parsing and verification; no duplicate campaign-report or request-attribution owner.
  • No paired RFC changed.
  • UI impact is none.
  • Commit includes DCO sign-off.

Signed-off-by: shangzh0 <97216392+shangzh0@users.noreply.github.com>

@songoow songoow left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · GPT-6 family · OpenAI · self_reported

动机 / Motivation

维护研究证据包的使用者在登记长 Codex 会话、刷新已有用量来源时,会经过这条本地观察路径。原实现同时保留整份字节、解码文本和分行副本;改后逐行解析并累积原始字节哈希,刷新来源时分块计算哈希。在相同合成输入下,登记、重放和历史读取结果保持一致,大捕获登记的 Python 分配峰值下降。

本次验收范围是事后轨迹观察的文件读取开销;求解器运行、checkpoint、模型输入 token 和原生评分的改进仍须各自验证。这个问题可以从旧实现的整文件复制直接定位,并由相同输入下的真实登记路径复现;保留原实现会使观察操作承担随捕获文本增长的额外内存。因此,这是一项有独立价值、可单独验证和回滚的修复。

改动思路 / Approach

原有 Python adapter 继续负责文件读取,TypeScript owner 继续负责会话完整性与计数归约;两个私有 IO helper 不增加第二套业务规则。当前 PR 的交付边界是 _usage_capture 的 JSONL 读取与 _captured_view 的原始来源复核。入口仍是现有 trajectory-pack-register,随后进入原有 TypeScript 计数 owner、pack 持久化和查询;没有新增状态、缓存、协议版本或配置。源文件包含模型正文时,正文仍经过解析后丢弃,传给计数 owner 的内容仍受原 allowlist 限制。

我也比较了相邻 quota reader:它服务活跃会话的累计观察,并有允许未完成尾记录的语义,不能直接替换完整冻结捕获的严格读取。这里保留各自的业务边界、仅在原 provider 内抽取两个 IO helper,范围合适。

具体改动 / Changes

本次完整 diff 为两个文件,生产代码 +23/-9、测试 +83。规范依据是 docs/reference/trajectory-assessment.md#solver-capture-registration,spec revision:04c5799b88f6c4cf12a224b7fa679f272e1e794e(固定版本)。以下 SC 索引对应该未编号章节的现有条款,未新增要求。

条款 当前 head 的核对结果
SC-1 来源路径、会话绑定与原始哈希 implemented:保留 containment 和会话校验;UTF-8、CRLF、无末尾换行得到原始字节 SHA-256。
SC-2 行号与累计计数 implemented:逐行编号后进入原有 reducer,完整 CLI 对照与 4,500-Turn 测试保留计数和来源范围。
SC-3 只读观察权限 implemented:登记和读回保持无分析 job、无执行 receipt;不增加 solver 或 quota 权限。
SC-4 历史来源读回 implemented:同大小、同 mtime 的改写仍识别为 changed;删除后保留已捕获投影并标为 unavailable。
SC-5 可选字段省略 implemented:省略用量参数保持原身份和响应形状;其他捕获变化不激活已有 capture-free pack。
SC-6 大记录传输边界 implemented:继续使用现有 bounded local snapshot;实际大计数历史通过,越界仍明确拒绝。

关键代码讲解

  1. _usage_rows 在二进制行上先更新哈希,再解码和解析 JSON。因此哈希覆盖原始换行字节,行号保持真实物理位置;坏记录仍会使登记失败。
  2. _usage_capture 替换输入遍历和最终 digest 来源。原 allowlist、完整会话默认值、显式 incomplete 模式以及 TypeScript 归约调用都保留。events 仍保存需要的计数事件,这是其既有内存边界。
  3. _source_digest / _captured_view 使用标准库分块哈希;读回分支 对主捕获和可选 input artifacts 都重新读全部字节,保留 current/historical 的判断和历史投影。

新增测试覆盖 UTF-8/CRLF、无末尾换行、坏 JSON、同大小同时间戳改写、32 MiB 合成文件分配上界和真实计数 owner。独立执行三份相关测试文件:PASS,54 passed in 51.72s。Python compile、精确 base/head git diff --check、diff-scoped semantic advisory 也通过;advisory 在一个变更生产文件中发现 0 个受支持的新词汇载体,仍以人工契约核对为准。

对主干的风险 / Risks

最需要防止的是优化导致哈希或行号变化、坏尾记录被跳过、旧来源被误报 current。补充验证使用相同合成输入,经真实 CLI、真实本地 TypeScript 后端、持久化和单独读回,对比 base 04c5799b88f6c4cf12a224b7fa679f272e1e794e 与 head 99eb92496edd023c24817a1e023d5673360356c1;19 项完整观察一致,包括遗漏参数、重放、错误 session、路径/软链越界、坏尾记录与错误 session 同时出现、incomplete 的默认拒绝与显式恢复,以及来源改写和删除。没有增加模型调用。

另用 33,628,287-byte 合成捕获,在等价预热后测量完整登记操作的 Python 分配峰值:

同一真实登记路径 base head
tracemalloc 峰值 103,927,537 bytes 3,306,527 bytes
同一 8 MiB 验证断言 FAIL,复现旧复制开销 PASS
计数、哈希、诊断与持久化读回 与 head 相同 与 base 相同

该 8 MiB 是本次合成探针的断言,不是项目新增的性能限额;测量不包含 Node RSS 或预热成本。单条巨大 JSON 记录和保留的计数事件仍影响内存,native request payload 与引用文本校验也有未改变的整文件读取。这些边界应继续按各自路径衡量。本地首轮被执行环境的 socket 权限挡住,保留记录后在允许本地通信的隔离环境重跑,完整 54 项通过;没有以跳过断言处理。

当前 PR 未增加 actor、authority、共享词汇或机器义务,默认关闭分支通过 base/head 对照。已观察 CI 有成功和 pending 项,没有把 pending 写成通过;当前代码判断有独立本地覆盖,最终 merge readiness 仍由其原有规则决定。

我的整体评价 / Verdict

APPROVE,当前 head 未发现可复现、需作者修改的正确性或资源回归。 这项改动达成了限定的观察路径内存目标,生产调用实际可达,两个 helper 的维护成本与问题相称。long_horizon 判断为 preserved:冻结身份、重放、历史证据与无隐式执行边界均保留;user_experience 在所测登记任务上 improved:同一操作获得同一结果,Python 分配减少,未增加交互步骤。原有词汇和 TypeScript 业务 owner 被复用。

本 PR 不改变页面、排版或交互文本;已检查的变更路径是 trajectory_assessment_store.py 及其 CLI 注册与读取调用。无本 PR 涉及的视觉范围。本次结论限定在上述源版本、真实路径与合成负例;solver 效率、端到端模型效果及总进程内存仍分别验证。发布前还应以 unchanged head 的最新读取为准,审批结论不代替 merge 资格。

English verdict: APPROVE — head 99eb924; no actionable findings. The 54-test suite, immutable base/head CLI parity, and allocation regression check passed. Solver and checkpoint performance remain outside this change.

@huangruiteng
huangruiteng merged commit 2650f8e into loopx-project:main Oct 31, 2026
33 of 39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants