Skip to content

Share duplicated R-runner, security-filter and schema definitions - #123

Merged
soodoku merged 2 commits into
mainfrom
dry-shared-definitions
Oct 23, 2026
Merged

soodoku merged 2 commits into
mainfrom
dry-shared-definitions

Conversation

@soodoku

@soodoku soodoku commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

Removes the copies a jscpd scan found in rmcp/ (renamed-identifier matching, ≥12 lines). jscpd goes from 17 clones to 0. The behavior doesn't change, except in the three places where the copies had already drifted apart (listed below).

Most of the r_integration.py diff is re-indentation: each runner lost one try: level once cleanup moved into a context manager. Review it with whitespace hidden (?w=1), which brings the whole PR down to +357/−661.

What was duplicated and where it lives now

Copies (line numbers on main) Now
tools/flexible_r.py 42–55 (DANGEROUS_PATTERNS, unused) vs 141–154 (inline remaining_dangerous, the list actually checked) validate_r_code checks DANGEROUS_PATTERNS. This is a security filter, and the module constant was dead code: editing it had no effect.
r_integration.py 268–284 vs 485–503, plus the cleanup finally blocks: temp script/args/result files, Windows path normalization, unlink _r_temp_files(args) context manager yielding _RTempFiles
r_integration.py 314–326 vs 661–673: failure report (R path, env, stdout/stderr) _r_failure_report(...), using _r_environment()
r_integration.py 335–362 vs 682–711: "there is no package called" handling and the package→feature map _missing_package_message(stderr) + _PACKAGE_FEATURES. The control flow around it stays per runner.
r_integration.py 1116–1142 vs 1210–1236, 1156–1168 vs 1257–1269: diagnosis dict and parsing of each check's output _new_diagnosis(), _record_r_version(...), _record_jsonlite_test(...), _JSONLITE_TEST_SCRIPT. The subprocess.run / asyncio.create_subprocess_exec calls stay separate.
tools/econometrics.py 40–57, 153–170; tools/regression.py 56–73: std_errors / t_values / p_values / r_squared core/schemas.py: coefficient_inference_schema(), r_squared_schema(), next to table_schema() / formula_schema()
tools/fileops.py 79–97 vs 597–615 vs 697–715: read summary block _read_summary_schema(). Each tool keeps its own required list.
tools/fileops.py 142–168 vs 220–246 (and the same props in write_json): write-result props _write_result_properties(), spread into each tool's own properties. required stays per tool.
tools/visualization.py 78–90 ×6: dimensions / image_data / image_mime_type _image_output_properties(). required stays per tool.
tools/visualization.py 102–114 ×6: return_image/width/height + execute_r_script_with_image_async _render_plot(context, params, r_script, default_width, default_height), which keeps each tool's own default size. Path confinement and get_r_script stay outside the try in each tool, as before, so their errors still skip the "creation failed" log.

Fragments are functions that return fresh dicts, not shared module-level dicts, so no two tool schemas share a mutable object. This matches the existing helpers in core/schemas.py.

Drifted copies: which version I kept

  1. Missing-package feature map. The async copy lacked knitr. Both runners now use the 19-entry map that includes "knitr": "Table Formatting & Reporting".
  2. Missing-package message. I kept the async wording (R -e "install.packages('pkg')") for both runners. The sync copy also printed a hard-coded "install all RMCP packages" list of 19 packages. I dropped it: it is a fraction of the 429-package allowlist and was a third copy of the map's keys.
  3. r_squared description. I kept "R-squared (coefficient of determination)". It replaces "R-squared (overall fit)" in panel_regression and "R-squared value" in instrumental_variables. Only the wording is unified. instrumental_variables keeps its schema without minimum: 0, because a 2SLS R-squared is negative when the instrumented fit is worse than the mean, and adding the bound would make output validation reject valid results. Its adj_r_squared wording ("Adjusted R-squared value") is untouched.

Schema guard

New tests/unit/tools/test_tool_schema_snapshot.py collects every @tool in rmcp.tools (50 tools) and compares each input and output schema with tests/fixtures/tool_schemas.json. I generated the fixture from main before refactoring. A comparison of the main-generated and branch-generated schemas shows exactly two differences, the r_squared descriptions above:

.instrumental_variables.output.properties.r_squared.description : "R-squared value" -> "R-squared (coefficient of determination)"
.panel_regression.output.properties.r_squared.description : "R-squared (overall fit)" -> "R-squared (coefficient of determination)"

I confirmed the test fails when it should: changing one description inside _read_summary_schema() makes it fail on read_csv. After an intended schema change, regenerate the fixture with uv run python tests/unit/tools/test_tool_schema_snapshot.py.

Checks (macOS, R 4.6.0 installed)

Command main branch
uv run ruff check . pass pass
uv run ruff format --check . pass pass
uv run pyright 0 errors 0 errors
uv run pytest 419 passed, 28 skipped 420 passed, 28 skipped (+1 snapshot test)
Rscript rmcp/r_assets/run_tests.R — pass
uv run --group docs sphinx-build -W -b html docs … — pass
uvx --python 3.12 --from preen==0.7.1 preen check --strict --skip tests exit 1 exit 1, same findings
  • Skips are identical on main and branch. None are due to R: 12 need a BuildKit Docker build, 7 need Claude Desktop, and 8 are marked aspirational.
  • The preen failure is on main too. It comes from two lockfile audit findings, certifi 2026.7.22 (PYSEC-2026-4320) and virtualenv 21.14.7 (CVE-2026-113722). Both are unrelated to this change and left for dependabot.

jscpd (uvx jscpd==5.4.2 rmcp --format python --ignore-identifiers=consistent --min-lines 12 --min-tokens 1 --reporters console): before 17 clones (289 duplicated lines), after 0.

Pre-existing bug, not fixed here

_missing_package_message matches there is no package called '([^']+)', but R under a UTF-8 locale prints curly quotes (‘knitrr’). I checked: on main and on this branch, a missing package gets the generic failure report, not the friendly message. Fixing that changes user-visible behavior, so it belongs in its own PR.

🤖 Generated with Claude Code

- flexible_r: validate_r_code checked an inline copy of the blocked R
  patterns while the module-level DANGEROUS_PATTERNS sat unused; it now
  checks the constant, so there is one list to keep current.
- r_integration: the sync and async runners share temp-file setup and
  cleanup (_r_temp_files), the failure report (_r_failure_report) and the
  missing-package explanation (_missing_package_message, _PACKAGE_FEATURES).
  The two diagnosis functions share their initial dict and the parsing of
  each check's output; the subprocess calls stay separate.
- core/schemas: coefficient_inference_schema() and r_squared_schema() for
  linear_model, panel_regression and instrumental_variables.
- fileops: _read_summary_schema() and _write_result_properties().
- visualization: _image_output_properties() and _render_plot().

Drifted copies, now one version each:
- missing-package map: the async copy lacked knitr; it has it now.
- missing-package message: the async wording (R -e "install.packages(...)")
  is used by both runners; the sync copy's hard-coded "install all" list
  of 19 packages is gone.
- r_squared description: "R-squared (coefficient of determination)" for
  linear_model, panel_regression and instrumental_variables. IV keeps its
  missing lower bound, since a 2SLS R-squared can be negative.

tests/unit/tools/test_tool_schema_snapshot.py pins every tool's input and
output schema to tests/fixtures/tool_schemas.json, generated from main;
the only fixture change from main is the two r_squared descriptions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T20:45:43.740575Z 5b38ccb New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@soodoku
soodoku merged commit 4be3013 into main Oct 23, 2026
14 checks passed
@soodoku
soodoku deleted the dry-shared-definitions branch October 23, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant