Skip to content

Give --js code the named groups to close #799 #806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mathiasrw merged 1 commit into from
Oct 23, 2026
Merged

Give --js code the named groups to close #799 #806

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -366,7 +366,7 @@ If planned renames clash, the process is halted and no change is applied to any
| `-g` | **`--glob-pipe`** Each line of the piped data is a path or a glob of the files to work on. They are used together with the ones given as arguments. [boolean] |
| | **`--null-in`** The piped paths and globs of --glob-pipe are separated by null bytes, like from `find -print0` or --null-out. [boolean] |
| `-R` | **`--replacement-pipe`** Replacement will be piped in. You still need to provide a dummy value (like `_`) as replacement parameter. [boolean] |
| `-j` | **`--replacement-js`**, **`--js`** Treat replacement as javascript source code. The statement from the last expression will become the replacement string. Purposefully implemented the most insecure way possible to remove _any_ incentive to consider running code from an untrusted part. The full match will be available as a javascript variable named $0 while each captured group will be available as $1, $2, $3, ... and so on. At some point, the $ char _will_ give you a headache when used from the command line, so use €0, €1, €2, €3... instead. If the javascript source code references to the full match or a captured group the code will run once per match. Otherwise, it will run once per file. The code has access to the following variables: `r` as an alias for `require` with both expanded to understand a relative path even if it is not starting with `./`, `fs` from node, `path` from node, `globs` from npm, `pipe`: the data piped into the command with --js-pipe (null if no piped data, and `pipe_` is `❌ `), `find`: pattern searched for (the needle), `text`: full text being searched i.e. file content or piped data (the haystack), `bytes`: total size of the haystack in bytes, `size`: human-friendly representation of the total size of the haystack, `time`: String representing the local time when the command was invoked, `time_obj`: date object representing `time`, `now`: alias for `time`, `cwd`: current process working dir, `nl`: a new-line char, `_`: a single space char (for easy string concatenation). The following values defaults to `❌` if haystack does not originate from a file: `file`: contains the full path of the active file being searched (including full filename), `file_rel`: contains `file` relative to current process working dir, `dirpath`: contains the full path without filename of the active file being searched, `dirpath_rel`: contains `dirpath` relative to current process working dir, `filename`: is the full filename of the active file being searched without path, `name`: filename of the active file being searched with no extension, `ext`: extension of the filename including leading dot, `mtime`: ISO inspired representation of the last local modification time of the current file, `ctime`: ISO representation of the local creation time of the current file. `mtime_obj`: date object representing `mtime`, `ctime_obj`: date object representing `ctime`. The numbers: `$n`: the number of the file, `$t`: the number of files, `$i`: the number of the file among the files with a match so far, `$c`: among the files changed so far, `$m`: the number of the match in the file, `$M`: across all files. Each has cousins with zeros in front (`$n0` has 2 digits, `$n00` 3, and so on to 10) and with a space at the end (`$n_`). All variables, except from module, date objects, `nl` and `_`, has a corresponding variable name followed by `_` where the content has an extra space at the end (for easy concatenation). [boolean] |
| `-j` | **`--replacement-js`**, **`--js`** Treat replacement as javascript source code. The statement from the last expression will become the replacement string. Purposefully implemented the most insecure way possible to remove _any_ incentive to consider running code from an untrusted part. The full match will be available as a javascript variable named $0 while each captured group will be available as $1, $2, $3, ... and so on. At some point, the $ char _will_ give you a headache when used from the command line, so use €0, €1, €2, €3... instead. If the javascript source code references to the full match or a captured group the code will run once per match. Otherwise, it will run once per file. The code has access to the following variables: `r` as an alias for `require` with both expanded to understand a relative path even if it is not starting with `./`, `fs` from node, `path` from node, `globs` from npm, `pipe`: the data piped into the command with --js-pipe (null if no piped data, and `pipe_` is `❌ `), `find`: pattern searched for (the needle), `text`: full text being searched i.e. file content or piped data (the haystack), `bytes`: total size of the haystack in bytes, `size`: human-friendly representation of the total size of the haystack, `time`: String representing the local time when the command was invoked, `time_obj`: date object representing `time`, `now`: alias for `time`, `cwd`: current process working dir, `nl`: a new-line char, `_`: a single space char (for easy string concatenation). The following values defaults to `❌` if haystack does not originate from a file: `file`: contains the full path of the active file being searched (including full filename), `file_rel`: contains `file` relative to current process working dir, `dirpath`: contains the full path without filename of the active file being searched, `dirpath_rel`: contains `dirpath` relative to current process working dir, `filename`: is the full filename of the active file being searched without path, `name`: filename of the active file being searched with no extension, `ext`: extension of the filename including leading dot, `mtime`: ISO inspired representation of the last local modification time of the current file, `ctime`: ISO representation of the local creation time of the current file. `mtime_obj`: date object representing `mtime`, `ctime_obj`: date object representing `ctime`. The numbers: `$n`: the number of the file, `$t`: the number of files, `$i`: the number of the file among the files with a match so far, `$c`: among the files changed so far, `$m`: the number of the match in the file, `$M`: across all files. Each has cousins with zeros in front (`$n0` has 2 digits, `$n00` 3, and so on to 10) and with a space at the end (`$n_`). A named group is a variable with `_` in front: `_year` for `(?<year>...)`, and `_year_` has a space at the end. All variables, except from module, date objects, `nl` and `_`, has a corresponding variable name followed by `_` where the content has an extra space at the end (for easy concatenation). [boolean] |
| `-x` | **`--exclude-re`** Files with a path that matches this regular expression will be excluded from processing. Can be provided multiple times. [string] |
| `-X` | **`--exclude-glob`** Files found with this glob will be excluded from being processed. Can be provided multiple times and will act as one group of globs, meaning you can use ! notation to negate. [string] |
| `-h` | **`--help`** Display help. [boolean] |
Expand Down
4 changes: 3 additions & 1 deletion src/cli.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,9 @@ The following values defaults to \`❌\` if haystack does not originate from a f
\`mtime_obj\`: date object representing \`mtime\`,
\`ctime_obj\`: date object representing \`ctime\`.

The numbers: \`$n\`: the number of the file, \`$t\`: the number of files, \`$i\`: the number of the file among the files with a match so far, \`$c\`: among the files changed so far, \`$m\`: the number of the match in the file, \`$M\`: across all files. Each has cousins with zeros in front (\`$n0\` has 2 digits, \`$n00\` 3, and so on to 10) and with a space at the end (\`$n_\`).
The numbers: \`$n\`: the number of the file, \`$t\`: the number of files, \`$i\`: the number of the file among the files with a match so far, \`$c\`: among the files changed so far, \`$m\`: the number of the match in the file, \`$M\`: across all files. Each has cousins with zeros in front (\`$n0\` has 2 digits, \`$n00\` 3, and so on to 10) and with a space at the end (\`$n_\`).

A named group is a variable with \`_\` in front: \`_year\` for \`(?<year>...)\`, and \`_year_\` has a space at the end.

All variables, except from module, date objects, \`nl\` and \`_\`, has a corresponding variable name followed by \`_\` where the content has an extra space at the end (for easy concatenation).
`
Expand Down
28 changes: 25 additions & 3 deletions src/engine.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -778,6 +778,15 @@ function countFile(config, text, result) {
}
}

// The code uses a named group of the pattern, like _year for (?<year>…)
function usesNamedGroups(config) {
return [...config.regex.source.matchAll(/\(\?<([A-Za-z_$][\w$]*)>/g)].some(([, name]) =>
new RegExp('(^|[^\\w$])_' + name.replace(/\$/g, '\\$') + '(?![\\w$])').test(
config.replacementOri
)
);
}

// Code that makes the numbers and their cousins, put in front of the code of --js.
// $n0 is $n with zeros in front to 2 digits, $n00 to 3 and so on to 10, and the ones with _ at the end have a space at the end.
function numbersCode(numbers) {
Expand Down Expand Up @@ -932,8 +941,12 @@ function dynamicReplacement(

// Run only once if no captured groups (replacement cant change).
// The result is given back from a function, so `$&` and `$$` in it are inserted as they are and not expanded.
// $m and $M are the number of the match, so they also need the code to run once per match
if (!/\$\d/.test(_config_rr.replacement) && !re.matchNumbers.test(_config_rr.replacement)) {
// $m, $M and the named groups are about the match, so they also need the code to run once per match
if (
!/\$\d/.test(_config_rr.replacement) &&
!re.matchNumbers.test(_config_rr.replacement) &&
!usesNamedGroups(_config_rr)
) {
const result = dynamicContent(
require,
fs,
Expand Down Expand Up @@ -1016,9 +1029,18 @@ function dynamicReplacement(
_match_rr++;
_config_rr.numbers.matches++;
var capturedGroups = numbersCode({m: _match_rr, M: _config_rr.numbers.matches});
for (var i = 0; i < arguments.length - 2; i++) {
// The match and the groups are followed by the offset and the full text, and by the named groups if the pattern has any
const named = arguments[arguments.length - 1];
const hasNamed = 'object' === typeof named && null !== named;
for (var i = 0; i < arguments.length - (hasNamed ? 3 : 2); i++) {
capturedGroups += 'var $' + i + '=' + JSON.stringify(arguments[i]) + '; ';
}
// A named group is a variable with _ in front, like _year for (?<year>…), and _year_ has a space at the end
if (hasNamed) {
for (const [name, value] of Object.entries(named)) {
capturedGroups += `var _${name} = ${JSON.stringify(value)}, _${name}_ = _${name} + ' '; `;
}
}

return dynamicContent(
require,
Expand Down
25 changes: 25 additions & 0 deletions test/fix/799.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// https://github.com/mathiasrw/rexreplace/issues/799
// Named groups had no variable in --js, and a pattern with one gave the position of the match as an extra group variable.
import {describe, test, expect} from 'bun:test';
import {rr} from '../helper.js';

describe('#799 groups in --js', () => {
test('only the groups of the pattern get a variable, also with a named group', async () => {
const config = {pipedData: 'ab', replacementJs: true};
expect((await rr('(?<x>a)(b)', '$1 + $2 + typeof $3', config)).out).toBe('abundefined');
});

test('a named group is a variable with _ in front, and _ at the end has a space', async () => {
const config = {pipedData: '2024-05 1999-12', replacementJs: true};
const pattern = '(?<year>\\d{4})-(?<month>\\d\\d)';
expect((await rr(pattern, '_month + "/" + _year', config)).out).toBe('05/2024 12/1999');
expect((await rr(pattern, '_month_ + _year', config)).out).toBe('05 2024 12 1999');
});

test('a named group that took no part in the match is undefined', async () => {
const config = {pipedData: 'ab', replacementJs: true};
expect((await rr('(?<x>a)|(?<y>b)', 'String(_x) + String(_y) + ","', config)).out).toBe(
'aundefined,undefinedb,'
);
});
});
Loading