Repository navigation
Implement $fy migration for shell and competitor scripts (JS + Rust) - #133
Conversation
Adding CLAUDE.md with task information for AI processing. This file will be removed when the task is complete. Issue: #1
- Add $fy virtual command that converts shell scripts to command-stream mjs format - Support stdin input for single commands and file input for full scripts - Handle shell operators: &&, ||, ; with proper JavaScript equivalents - Convert pipelines, sequences, variables, exports, and comments - Preserve script structure while translating to command-stream syntax - Add comprehensive test suite with 14 test cases - Include example scripts and debug utilities Features: - Single command conversion: echo "ls -la" | $fy - File conversion: $fy script.sh [output.mjs] - Shell operator translation (&&, ||, ;) - Pipeline preservation - Comment and structure preservation - Variable and export handling - Error handling for missing files 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
konard
changed the title
|
We should use https://github.com/link-foundation/meta-language, and may be logic from https://github.com/link-foundation/relative-meta-logic, in order to actually implement rule based translation, where we first formalize into meta language, and after it use substitution rules on top of meta language to actually finish the translation. Any missing features from https://github.com/link-foundation/meta-language must be reported, so we can fix that first, before continuing this pull request. Also may be you will learn something from github.com/link-assistant/formal-ai related to that specific task. So we use all the best practices possible for both JavaScript and Rust. |
|
🤖 AI Work Session Started Starting automated work session at 2026-07-25T19:11:32.340Z The PR has been converted to draft mode while work is in progress. This comment marks the beginning of an AI work session. Please wait for the session to finish, and provide your feedback. |
The first draft of $fy translated shell to JavaScript with ad-hoc string munging. This replaces it with the two-stage pipeline requested in review: 1. `formalizeShell` parses the script and formalizes it into a link-foundation/meta-language `LinkNetwork` of typed syntax links (`LinkType.Syntax`, language `Shell`). 2. `buildRuleSet` declares Shell -> JavaScript `TranslationRule`s and a small engine substitutes them over the network, rendering each node in the sub-language its parent asks for (statement, command, value, expression). Missing meta-language features found while doing this are reported in docs/meta-language-gaps.md so they can be fixed upstream. Also fixes two genuine bugs found on the way: - `parseSimpleCommand` treated reserved words as block terminators in any position, so `echo done | tr a-z A-Z` lost its argument. Reserved words are only reserved in command position. - `getStdinData` returned the stdio *mode* string as stdin *data*, so every virtual command saw the literal text `inherit` on stdin. experiments/fy-pipeline-repro.sh reproduces the parser bug and examples/fy-translate.mjs demonstrates the whole pipeline.
The reviewer asked for the best practices of both languages, so the
rule-based translator now exists in Rust as well: rust/src/fy mirrors
js/src/fy stage for stage (formalize into a meta-language links network,
then substitute through a TranslationRuleSet).
fixtures/fy/ holds one shell script and the module both implementations
must produce from it; rust/tests/fy.rs and js/tests/fy-tool.test.mjs
assert against those bytes, so the two cannot drift apart.
Also translate ${1:-default}: the brace expander only accepted names, so
positional and special parameters with a default fell through to the
untranslated \${...} escape hatch.
Reports two further meta-language findings: the Rust crate has no public
dynamic-arity link insertion (worked around by chunking), and
with_reference_capture exists in Rust 0.54 but not in JS 0.46.
Done. 1. Formalize → substitute
2. Missing meta-language features — reportedFull write-up with reproductions in
Gaps 1–3 block rule-based translation entirely, so Also confirmed on the Rust side by reading the crate: 3. JavaScript / Rust parity
|
konard
changed the title
The Node.js compatibility job imports js/src/$.mjs without installing dependencies, and the $fy command pulled meta-language in statically, so the module no longer loaded on Node: Module failed to load: Cannot find package 'meta-language' imported from js/src/fy/shell-formalizer.mjs Importing the translator inside the command handler keeps meta-language out of the library's import graph until $fy actually runs.
The Windows job failed on both new tests. A script checked out with CRLF
endings reached the lexer with the \r attached to each word, which broke
the parse ('rule `for` has no JavaScript:command template'), so both
translators now normalize line endings first. The golden fixture must
also arrive with LF for a byte-for-byte comparison to mean anything,
which .gitattributes now enforces.
The run-the-translation test passed a URL pathname as a module
specifier; on Windows that is '/D:/...', which is not importable, so it
passes the file URL instead.
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost: $29.524047📊 Context and tokens usage:Claude Opus 5: (7 sub-sessions)
Total: (16.2K new + 593.9K cache writes + 34.2M cache reads) input tokens, 256.4K output tokens, $29.524047 cost 🤖 Models used:
📎 Log file uploaded as Gist (10705KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
✅ Ready to mergeThis pull request is now ready to be merged:
Monitored by hive-mind with --auto-restart-until-mergeable flag |
|
Double check we fully deliver all the requirements and also make sure we have translators from source code of our competitors to our source code, as well as automatic translation for shell scripts into mjs scripts that uses our library + use-m. #133 (comment) - double check all from here also delivered. We don't wait until latest version of meta-language will be fully done, yet we can report issues if we have something missing there. And all conflicts must be resolved. |
|
🤖 AI Work Session Started Started at 2026-10-24T08:07:56.165Z. Please wait for it to finish before giving feedback. Runtime: solve |
konard
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Working session summaryPR #133 is updated and ready for review. Shell and competitor translators are implemented in JavaScript and Rust, and merge conflicts are resolved. Passed: 2,871 JavaScript tests, 958 Rust tests, 124 Node regressions, and all 49 CI checks. The working tree is clean. This summary was automatically extracted from the AI working session output. |
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost estimation:
📊 Context and tokens usage:
Total: (566.4K + 26.0M cached) input tokens, 156.3K output tokens, $9.812416 cost 🤖 Models used:
📎 Log file uploaded as Gist (5065KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
$fynow converts shell scripts and competitor JavaScript into executable command-stream modules in both JavaScript and Rust. Shell translation automatically loads command-stream throughuse-m; competitor translation uses the matching command-stream compatibility entry points.Fixes #1
Resulting behavior
await use('command-stream');--no-use-msupports an already installed library. File translation retains the source basename for$0.command-stream/*entry points while preserving surrounding source..cjs/--commonjsproduces an executable ES module usingcreateRequire, preserving top-levelthisand exports identity.LinkNetworksyntax/token links, then applyLinkQueryandTranslationRuleSetsubstitutions. The structural shell compiler remains available with--structural; current main's rootfy()helper and compatibility APIs remain intact.require; comments, ordinary strings, regexes and property calls are preserved. Output files cannot overwrite existing files.main, including the lightweight core and Rust 1.10.1 release.$fyis registered in the shared core without loading the compiler eagerly; cold ESM/CommonJS entry points remain within the 100-module budget. Updated CLI/API documentation and executable examples, and retained minor-release changeset/changelog triggers for both packages.Review requirements
The July review and October follow-up are addressed by the formalize/substitute implementation, competitor migrations, default use-m output and resolved merge. Representation/evaluation separation and composable rules were informed by relative-meta-logic and formal-ai; neither adds a runtime dependency.
Rechecked published meta-language packages: npm
latestremains 0.46.0; Rust is now 1.0.0. Rust's public dynamic-arity insertion replaces the old chunk-node workaround. Remaining evaluator differences are documented with executable probes in the integration findings and reported in the upstream recheck. Translation works independently of further upstream releases. Rust meta-language 1.0.0 still pins tree-sitter to 0.27.0 exactly; the existing freshness exception policy accepts the documented open upstream blocker.The latest main release run also exposed changeset merging before remote synchronization. Merging now happens inside the version script after synchronization. A real Git regression with two pending changesets and a concurrent Rust release reproduces the original dirty-tree rebase failure and passes with the corrected order. The downloaded main failure log records the exact error at
ci-logs/main-javascript-40081061680.log:51286.Reproduction and validation
fixtures/fy/competitors.jspreviously had no source migration path;fixtures/fy/sample.shdid not produce the requested use-m module. Their expected generated modules now have shared JavaScript/Rust regression tests.fixtures/fy/commonjs.cjsalso reproduces the formerly incorrect top-levelthisbinding; its generated module must preserve the original exports identity. Runtime tests execute all five competitor migrations in Node and Bun, compare shell functions/arguments/stdin/status with the original interpreter, and exercise the actual use-m loader and both JS CLI entry points. Native Rust tests cover CLI/stdin, virtual$fy, output safety and identical fixtures.Local validation:
All 49 PR checks pass on
f5c5ee813f369bdd94ff3fbec2263607171702ac, including Linux/macOS/Windows test matrices, Node 22/24/26 checks, release build and verified packaging, benchmarks, parity, dependency freshness and security. Failed-run logs were downloaded and checked against commit timestamps and SHAs: the benchmark locked-build failure (ci-logs/benchmarks-40078904274.log:639) reportedcannot update the lock file ... because --locked was passedand required refreshing its dependency graph; generated docs (ci-logs/parity-40078904294.log:805) reportedGenerated documentation is out of dateand required the updated virtual-command catalog; freshness (ci-logs/dependencies-40079434519.log:756) reported pending lockfile updates and required updating the benchmark's compatible smallvec version. Execa and fancy-regex were updated to their latest releases, and the resolved Execa type blocker was removed. CodeQL's shebang backtracking finding is fixed with a line parser, a bounded scaling probe and a regression test; GitHub now marks the alert fixed.See the translator guide for options, supported entry points and runtime requirements. JavaScript input means standard JavaScript (TypeScript should be compiled first); Bun migration targets its shell API. Keep migrated files beside their inputs when relative imports depend on that location.