diff --git a/README.md b/README.md index 0e3e1a62..eefacc19 100644 --- a/README.md +++ b/README.md @@ -332,12 +332,11 @@ If planned renames clash, the process is halted and no change is applied to any | | **`--rename-force`** With --rename or --rename-path, write over a file that is there already. [boolean] | | `-m` | **`--output-match`** Output each match on a new line. Will not replace any content but you still need to provide a dummy value (like `_`) as replacement parameter. If search pattern does not contain matching groups the full match will be outputted. If search pattern does contain matching groups only matching groups will be outputted (same line with no delimiter). [boolean] | | `-T` | **`--trim-pipe`** Trim piped data before processing. If piped data only consists of chars that can be trimmed (new line, space, tabs...) it will become an empty string. [boolean] | -| `-p` | **`--pipe`** Read the piped data and make it available to --js as `pipe`. [boolean] | -| | **`--js-pipe`** Shorthand for --js --pipe (-jp). [boolean] | +| | **`--js-pipe`** Treat the replacement as javascript, and give the code the piped data as `pipe`. [boolean] | | `-g` | **`--glob-pipe`** Each line of the piped data is a path or a glob of the files to work on. They are used together with the ones given as arguments. [boolean] | | | **`--null-in`** The piped paths and globs of --glob-pipe are separated by null bytes, like from `find -print0` or --null-out. [boolean] | | `-R` | **`--replacement-pipe`** Replacement will be piped in. You still need to provide a dummy value (like `_`) as replacement parameter. [boolean] | -| `-j` | **`--replacement-js`**, **`--js`** Treat replacement as javascript source code. The statement from the last expression will become the replacement string. Purposefully implemented the most insecure way possible to remove _any_ incentive to consider running code from an untrusted part. The full match will be available as a javascript variable named $0 while each captured group will be available as $1, $2, $3, ... and so on. At some point, the $ char _will_ give you a headache when used from the command line, so use €0, €1, €2, €3... instead. If the javascript source code references to the full match or a captured group the code will run once per match. Otherwise, it will run once per file. The code has access to the following variables: `r` as an alias for `require` with both expanded to understand a relative path even if it is not starting with `./`, `fs` from node, `path` from node, `globs` from npm, `pipe`: the data piped into the command with --pipe (null if no piped data, and `pipe_` is `❌ `), `find`: pattern searched for (the needle), `text`: full text being searched i.e. file content or piped data (the haystack), `bytes`: total size of the haystack in bytes, `size`: human-friendly representation of the total size of the haystack, `time`: String representing the local time when the command was invoked, `time_obj`: date object representing `time`, `now`: alias for `time`, `cwd`: current process working dir, `nl`: a new-line char, `_`: a single space char (for easy string concatenation). The following values defaults to `❌` if haystack does not originate from a file: `file`: contains the full path of the active file being searched (including full filename), `file_rel`: contains `file` relative to current process working dir, `dirpath`: contains the full path without filename of the active file being searched, `dirpath_rel`: contains `dirpath` relative to current process working dir, `filename`: is the full filename of the active file being searched without path, `name`: filename of the active file being searched with no extension, `ext`: extension of the filename including leading dot, `mtime`: ISO inspired representation of the last local modification time of the current file, `ctime`: ISO representation of the local creation time of the current file. `mtime_obj`: date object representing `mtime`, `ctime_obj`: date object representing `ctime`. All variables, except from module, date objects, `nl` and `_`, has a corresponding variable name followed by `_` where the content has an extra space at the end (for easy concatenation). [boolean] | +| `-j` | **`--replacement-js`**, **`--js`** Treat replacement as javascript source code. The statement from the last expression will become the replacement string. Purposefully implemented the most insecure way possible to remove _any_ incentive to consider running code from an untrusted part. The full match will be available as a javascript variable named $0 while each captured group will be available as $1, $2, $3, ... and so on. At some point, the $ char _will_ give you a headache when used from the command line, so use €0, €1, €2, €3... instead. If the javascript source code references to the full match or a captured group the code will run once per match. Otherwise, it will run once per file. The code has access to the following variables: `r` as an alias for `require` with both expanded to understand a relative path even if it is not starting with `./`, `fs` from node, `path` from node, `globs` from npm, `pipe`: the data piped into the command with --js-pipe (null if no piped data, and `pipe_` is `❌ `), `find`: pattern searched for (the needle), `text`: full text being searched i.e. file content or piped data (the haystack), `bytes`: total size of the haystack in bytes, `size`: human-friendly representation of the total size of the haystack, `time`: String representing the local time when the command was invoked, `time_obj`: date object representing `time`, `now`: alias for `time`, `cwd`: current process working dir, `nl`: a new-line char, `_`: a single space char (for easy string concatenation). The following values defaults to `❌` if haystack does not originate from a file: `file`: contains the full path of the active file being searched (including full filename), `file_rel`: contains `file` relative to current process working dir, `dirpath`: contains the full path without filename of the active file being searched, `dirpath_rel`: contains `dirpath` relative to current process working dir, `filename`: is the full filename of the active file being searched without path, `name`: filename of the active file being searched with no extension, `ext`: extension of the filename including leading dot, `mtime`: ISO inspired representation of the last local modification time of the current file, `ctime`: ISO representation of the local creation time of the current file. `mtime_obj`: date object representing `mtime`, `ctime_obj`: date object representing `ctime`. All variables, except from module, date objects, `nl` and `_`, has a corresponding variable name followed by `_` where the content has an extra space at the end (for easy concatenation). [boolean] | | `-x` | **`--exclude-re`** Files with a path that matches this regular expression will be excluded from processing. Can be provided multiple times. [string] | | `-X` | **`--exclude-glob`** Files found with this glob will be excluded from being processed. Can be provided multiple times and will act as one group of globs, meaning you can use ! notation to negate. [string] | | `-h` | **`--help`** Display help. [boolean] | diff --git a/devguide.md b/devguide.md index e02049e6..dc72729c 100644 --- a/devguide.md +++ b/devguide.md @@ -22,15 +22,16 @@ All output goes through `src/output.ts`. Pick the function by who the message is | `print(item, delimiter, tail, stream)` | Always | The result. Puts the delimiter after each item, or only between them without a tail. Without a delimiter it is a line break, unless the item already ends with one. | A name, the content of a file, a match | | `printDone()` | At the end of a run, also when it is stopped | Ends stdout and stderr with a line break when `--nl` is set | | | `info(name)` | Unless `--quiet` | The name of an affected file. stdout, or stderr when stdout has content. | `a.md` | -| `warn(msg)` | Unless `--quiet` | Something is likely a mistake, but the run goes on. Also added to `status.warnings`. | `Data is piped in, but no flag tells what to do with it` | +| `warn(msg)` | Unless `--quiet` | Something is likely a mistake, or there is nothing to do, but the run goes on. Starts with 🟡. Also added to `status.warnings`. | `0 files found` | | `error(msg, data)` | Unless `--quiet-total` | Something went wrong. Starts with ❌. The run goes on, unless `--halt` is set: then a panic is thrown. Also added to `status.errors`, so the CLI gives the exit code 2. | `File not found: a.md` | -| `panic(msg, data, help)` | Unless `--quiet-total` | Something went wrong and the run can not go on. Starts with ❌ and ends with `No further processing.` Throws a panic, and the CLI exits with 3. Also added to `status.errors`. | `No data piped into replacement` | +| `checkFailed(text)` | Unless `--quiet-total` | A check from `--expect-*` that is not met. An error, also in `status.checks`, so the CLI gives the exit code 1 when the failed checks are the only errors. | `Expected a diff, but nothing was changed` | +| `panic(msg, data, help)` | Unless `--quiet-total` | Something went wrong and the run can not go on. Starts with ❌, ends with `No further processing.`, and throws a panic, so the CLI exits with 3. Also added to `status.errors`. | `No data piped into replacement` | | `chat(msg)` | `--verbose`, also with `--quiet`, but not with `--quiet-total` | The user: what happens to each file. Only ever to stderr. Also kept in `engine.steps` of the status report. | `Open sync: a.md`, `Nothing changed in: a.md` | - What the run did is in the status report: the engine resolves with it, and `--status` prints it. So there is no extra debug output to print along the way. - `--debug` prints only the status report, with what went into the engine and what it did. -- `warn()`, `error()` and `panic()` add to the status report as they go, also when the quiet flags hide what they print. -- The exit code is 0 when the run is done, 2 when a file gave an error, and 3 for a panic or wrong flags. 1 is kept for a check that says no, like `grep`. See Exit codes in the README. +- `warn()`, `error()`, `checkFailed()` and `panic()` add to the status report as they go, also when the quiet flags hide what they print. +- The exit code is 0 when the run is done, 1 when a check from `--expect-*` is not met, 2 when a file gave an error, and 3 for a panic or wrong flags. See Exit codes in the README. ## Tests diff --git a/src/cli.ts b/src/cli.ts index 0614ed9f..58360644 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -312,12 +312,11 @@ const yargs = require('yargs') '' ) - .boolean('p') - .alias('p', 'pipe') - .describe('p', `Read the piped data and make it available to --js as \`pipe\`.`) - .boolean('js-pipe') - .describe('js-pipe', `Shorthand for --js --pipe (-jp).`) + .describe( + 'js-pipe', + `Treat the replacement as javascript, and give the code the piped data as \`pipe\`.` + ) .boolean('g') .alias('g', 'glob-pipe') @@ -326,7 +325,7 @@ const yargs = require('yargs') 'Each line of the piped data is a path or a glob of the files to work on. They are used together with the ones given as arguments.' ) // The piped data can only be one thing - .conflicts('g', ['R', 'p', 'js-pipe']) + .conflicts('g', ['R', 'js-pipe']) .boolean('null-in') .describe( @@ -352,7 +351,7 @@ The code has access to the following variables: \`fs\` from node, \`path\` from node, \`globs\` from npm, -\`pipe\`: the data piped into the command with --pipe (null if no piped data, and \`pipe_\` is \`❌ \`), +\`pipe\`: the data piped into the command with --js-pipe (null if no piped data, and \`pipe_\` is \`❌ \`), \`find\`: pattern searched for (the needle), \`text\`: full text being searched i.e. file content or piped data (the haystack), \`bytes\`: total size of the haystack in bytes, diff --git a/src/engine.ts b/src/engine.ts index 905895af..7d8faa54 100644 --- a/src/engine.ts +++ b/src/engine.ts @@ -108,7 +108,7 @@ async function run(config) { if (config.globPipe) { if (config.replacementPipe || config.pipe) { panic( - '--glob-pipe can not be used with --replacement-pipe or --pipe, as the piped data can only be one thing' + '--glob-pipe can not be used with --replacement-pipe or --js-pipe, as the piped data can only be one thing' ); } config.pipedDataUsed = true; @@ -521,18 +521,8 @@ function pipeType(config, piped) { } function handlePipedData(config) { + // With files the piped data is only read when a flag asks for it, and then the flag uses it if (config.includeGlob.length) { - // Empty piped data (or only a line break) is not worth a warning - if ( - !config.replacementJs && - !config.pipedDataUsed && - config.pipedData && - '\n' !== config.pipedData && - '\r\n' !== config.pipedData - ) { - warn('Data is piped in, but no flag tells what to do with it. It is not used.'); - } - return false; } diff --git a/test/cli/glob-pipe.test.js b/test/cli/glob-pipe.test.js index 959e175a..67198606 100644 --- a/test/cli/glob-pipe.test.js +++ b/test/cli/glob-pipe.test.js @@ -15,7 +15,7 @@ describe('--glob-pipe', () => { }); }); - test.each([['--replacement-pipe'], ['--pipe'], ['--js-pipe']])( + test.each([['--replacement-pipe'], ['--js-pipe']])( 'can not be used with %s, as the piped data can only be one thing', async flag => { fs.writeFileSync('my.file', 'foobar\n'); diff --git a/test/cli/pipe.test.js b/test/cli/pipe.test.js index a763a5ea..0e715b37 100644 --- a/test/cli/pipe.test.js +++ b/test/cli/pipe.test.js @@ -3,10 +3,8 @@ import fs from 'fs'; import {$} from 'bun'; import {cli, sh} from '../helper.js'; -const warning = ' 🟡 Data is piped in, but no flag tells what to do with it. It is not used.\n\n'; - -describe('--pipe and --js-pipe', () => { - test('without --pipe the code does not get the piped data', async () => { +describe('--js-pipe', () => { + test('without it the code does not get the piped data', async () => { fs.writeFileSync('my.file', 'foobar\n'); expect(await sh($`echo -n x | bun ${cli} foo pipe my.file -o --js`)).toEqual({ out: 'nullbar\n', @@ -15,16 +13,8 @@ describe('--pipe and --js-pipe', () => { }); }); - test('the code gets the piped data with --js --pipe', async () => { - fs.writeFileSync('my.file', 'foobar\n'); - expect((await sh($`echo -n x | bun ${cli} foo pipe my.file -o --js --pipe`)).out).toBe( - 'xbar\n' - ); - }); - - test('the code gets the piped data with -jp and --js-pipe', async () => { + test('gives the code the piped data as pipe', async () => { fs.writeFileSync('my.file', 'foobar\n'); - expect((await sh($`echo -n x | bun ${cli} foo pipe my.file -o -jp`)).out).toBe('xbar\n'); expect((await sh($`echo -n x | bun ${cli} foo pipe my.file -o --js-pipe`)).out).toBe('xbar\n'); }); @@ -34,39 +24,11 @@ describe('--pipe and --js-pipe', () => { 'xbar\n' ); }); -}); - -describe('the warning that piped data is not used', () => { - test('is given when data is read with --pipe, but not used', async () => { - fs.writeFileSync('my.file', 'foobar\n'); - expect(await sh($`echo -n x | bun ${cli} o x my.file --pipe`)).toEqual({ - out: 'my.file\n', - err: warning, - exit: 0, - }); - expect(fs.readFileSync('my.file', 'utf8')).toBe('fxxbar\n'); - }); - - test('is not given without --pipe, as the data is not read', async () => { - fs.writeFileSync('my.file', 'foobar\n'); - expect((await sh($`echo -n x | bun ${cli} o x my.file`)).err).toBe(''); - }); - - test('is not given with --quiet', async () => { - fs.writeFileSync('my.file', 'foobar\n'); - expect((await sh($`echo -n x | bun ${cli} o x my.file --pipe -q`)).err).toBe(''); - }); - - test('is not given when the data is used', async () => { - fs.writeFileSync('my.file', 'foobar\n'); - expect((await sh($`echo -n x | bun ${cli} b _ my.file -R`)).err).toBe(''); - expect((await sh($`echo -n x | bun ${cli} b pipe my.file --js-pipe`)).err).toBe(''); - }); - test('is not given when nothing, or only a line break, is piped in', async () => { + test.each([['--pipe'], ['-p']])('%s is gone, as --js-pipe does the same', async flag => { fs.writeFileSync('my.file', 'foobar\n'); - expect((await sh($`echo -n '' | bun ${cli} o x my.file --pipe`)).err).toBe(''); - expect((await sh($`echo '' | bun ${cli} o x my.file --pipe`)).err).toBe(''); - expect((await sh($`bun ${cli} o x my.file --pipe < ${new Response('\r\n')}`)).err).toBe(''); + const result = await sh($`echo -n x | bun ${cli} foo pipe my.file -o --js ${flag}`); + expect(result.exit).toBe(3); + expect(result.err).toContain('Unknown argument'); }); }); diff --git a/test/cli/run.sh b/test/cli/run.sh index ccd46478..e2130616 100644 --- a/test/cli/run.sh +++ b/test/cli/run.sh @@ -193,10 +193,6 @@ assert "printf x | rexreplace 'b' _ my.file -o --replacement-pipe" 'fooxa reset assert "printf x | rexreplace 'foo' 'pipe' my.file -o --js-pipe" 'xbar' -# Data that is piped in but not used gives a warning -reset -assert "printf x | rexreplace 'o' 'x' my.file --pipe 2>&1 >/dev/null | grep -c 'Data is piped in'" '1' - # The names of the changed files go to stdout, so they can be piped on reset diff --git a/test/engine/verbose.test.js b/test/engine/verbose.test.js index 8ce551d6..707a48a2 100644 --- a/test/engine/verbose.test.js +++ b/test/engine/verbose.test.js @@ -35,13 +35,6 @@ describe('--verbose', () => { ); }); - test('is not needed to be told that piped data is not used (a warning since #719)', async () => { - fs.writeFileSync('my.file', 'foobar'); - const config = {includeGlob: ['my.file'], pipedData: 'boo'}; - expect((await rr('o', 'x', {...config, verbose: true})).err).toContain('Data is piped in'); - expect((await rr('x', 'o', config)).err).toContain('Data is piped in'); - }); - test('never writes to stdout', async () => { expect((await rr('o', 'x', {pipedData: 'foobar', verbose: true})).out).toBe('fxxbar'); }); @@ -80,7 +73,7 @@ describe('--verbose', () => { verbose: true, replacementJs: true, }; - expect((await rr('foo', 'pipe', config)).err).not.toContain('Data is piped in'); + await rr('foo', 'pipe', config); expect(fs.readFileSync('my.file', 'utf8')).toBe('boobar'); }); }); diff --git a/test/fix/711.test.js b/test/fix/711.test.js index fa8bd22d..dc877428 100644 --- a/test/fix/711.test.js +++ b/test/fix/711.test.js @@ -79,7 +79,7 @@ describe('#711 --glob-pipe', () => { test.each([ ['--replacement-pipe', {replacementPipe: true}], - ['--pipe', {pipe: true}], + ['--js-pipe', {replacementJs: true, pipe: true}], ])('can not be used with %s, as the piped data can only be one thing', async (name, flags) => { fs.writeFileSync('a.file', 'foo'); const result = await rr('o', 'x', {pipedData: 'a.file', globPipe: true, ...flags}); diff --git a/test/fix/719.test.js b/test/fix/719.test.js deleted file mode 100644 index be96a67d..00000000 --- a/test/fix/719.test.js +++ /dev/null @@ -1,76 +0,0 @@ -// https://github.com/mathiasrw/rexreplace/issues/719 -// Data piped in together with files was not used, and nothing told about it. -import {describe, test, expect} from 'bun:test'; -import fs from 'fs'; -import {rr} from '../helper.js'; - -const warning = ' 🟡 Data is piped in, but no flag tells what to do with it. It is not used.\n\n'; -const expectedOutput = 'my.file\n'; - -describe('#719 a warning when piped data is not used', () => { - test('is given when data is piped in and files are also given', async () => { - fs.writeFileSync('my.file', 'foobar'); - expect(await rr('b', '_', {includeGlob: ['my.file'], pipedData: 'x'})).toEqual({ - out: expectedOutput, - err: warning, - exit: null, - }); - expect(fs.readFileSync('my.file', 'utf8')).toBe('foo_ar'); - }); - - test.each(['quiet', 'quietTotal'])('is not shown with --%s', async flag => { - fs.writeFileSync('my.file', 'foobar'); - expect(await rr('b', '_', {includeGlob: ['my.file'], pipedData: 'x', [flag]: true})).toEqual({ - out: '', - err: '', - exit: null, - }); - }); - - test('does not stop the run with --halt', async () => { - fs.writeFileSync('my.file', 'foobar'); - expect(await rr('b', '_', {includeGlob: ['my.file'], pipedData: 'x', halt: true})).toEqual({ - out: expectedOutput, - err: warning, - exit: null, - }); - expect(fs.readFileSync('my.file', 'utf8')).toBe('foo_ar'); - }); - - test('is not given when the piped data is used', async () => { - fs.writeFileSync('my.file', 'foobar'); - expect( - await rr('b', '_', {includeGlob: ['my.file'], pipedData: 'x', replacementPipe: true}) - ).toEqual({ - out: expectedOutput, - err: '', - exit: null, - }); - fs.writeFileSync('my.file', 'foobar'); - expect( - await rr('b', 'pipe', {includeGlob: ['my.file'], pipedData: 'x', replacementJs: true}) - ).toEqual({ - out: expectedOutput, - err: '', - exit: null, - }); - expect(fs.readFileSync('my.file', 'utf8')).toBe('fooxar'); - }); - - test.each([null, '', '\n', '\r\n'])('is not given when %j is piped in', async pipedData => { - fs.writeFileSync('my.file', 'foobar'); - expect(await rr('b', '_', {includeGlob: ['my.file'], pipedData})).toEqual({ - out: expectedOutput, - err: '', - exit: null, - }); - }); - - test('is not given when only data is piped in', async () => { - expect(await rr('b', '_', {pipedData: 'foobar'})).toEqual({ - out: 'foo_ar', - err: '', - exit: null, - }); - }); -}); diff --git a/test/fix/742.test.js b/test/fix/742.test.js index 0c9b8ff6..bc93c2a9 100644 --- a/test/fix/742.test.js +++ b/test/fix/742.test.js @@ -47,15 +47,8 @@ describe('#742 the engine resolves with a status', () => { }); test('with the warnings, also when --quiet hides them', async () => { - fs.writeFileSync('a.file', 'foo'); - const status = await rrStatus('o', 'x', { - includeGlob: ['a.file'], - pipedData: 'x', - quiet: true, - }); - expect(status.warnings).toEqual([ - 'Data is piped in, but no flag tells what to do with it. It is not used.', - ]); + const status = await rrStatus('o', 'x', {includeGlob: ['nope.file'], quiet: true}); + expect(status.warnings).toEqual(['0 files found']); }); });