Let the engine return a status report and the CLI own the process to close #742

devguide.md

@@ -10,6 +10,7 @@ How we like the code in RexReplace to be.
 - **Comments say why, not what.** Write them in plain, simple English. Leave the comment out if the code already says it.
 - **No sloppy shortcuts.** If two cases need different calls, write a plain `if`/`else` instead of one call that only half fits both.
 - **Everything is printed with `write()` in `src/output.ts`,** so `--nl` knows how each stream ends.
+- **The engine never touches the process.** It resolves with the status report of the run, or throws a panic when the run can not go on. Only `src/cli.ts` reads `argv` and stdin, sets the exit code and ends the process.
 - **stdout is the result.** That is the names of the affected files, or the content from `--output`, `--output-match` or piped data. Everything else goes to stderr, so the result can always be piped.
 
 ## Telling what is going on
@@ -21,22 +22,19 @@ All output goes through `src/output.ts`. Pick the function by who the message is
 | `print(item, delimiter, tail, stream)` | Always | The result. Puts the delimiter after each item, or only between them without a tail. Without a delimiter it is a line break, unless the item already ends with one. | A name, the content of a file, a match |
 | `printDone()` | At the end of a run, also when it is stopped | Ends stdout and stderr with a line break when `--nl` is set | |
 | `info(name)` | Unless `--quiet` | The name of an affected file. stdout, or stderr when stdout has content. | `a.md` |
-| `warn(msg)` | Unless `--quiet` | Something is likely a mistake, but the run goes on | `Data is piped in, but no flag tells what to do with it` |
-| `error(msg, data)` | Unless `--quiet-total` | Something went wrong. The run goes on, unless `--halt` is set. | `File not found: a.md` |
-| `die(msg, data, help)` | Unless `--quiet-total` | Something went wrong and the run can not go on. Exits with 1. | `No data piped into replacement` |
-| `chat(msg)` | `--verbose` (or `--debug`) | The user: what happens to each file | `Open sync: a.md`, `Nothing changed in: a.md` |
-| `debug(data)` | `--debug` | The developer: what happens inside, at points that matter | `Work on content from: a.md` |
-| `step(data)` | `--verbose` and `--debug` | The developer: a trace of every step, with the data at that step | `Get final regex`, the full config |
-
-- The three levels of telling go from little to a lot: `chat`, then `debug`, then `step`.
-- `chat` falls back to `debug` without `--verbose`, so `--debug` alone also shows what happens to each file.
-- `debug` and `step` print their data as JSON. That keeps line breaks and spaces visible.
-- `chat`, `debug` and `step` only ever write to stderr.
+| `warn(msg)` | Unless `--quiet` | Something is likely a mistake, but the run goes on. Also added to `status.warnings`. | `Data is piped in, but no flag tells what to do with it` |
+| `error(msg, data)` | Unless `--quiet-total` | Something went wrong. The run goes on, unless `--halt` is set: then a panic is thrown. Also added to `status.errors`, so the CLI gives the exit code 1. | `File not found: a.md` |
+| `die(msg, data, help)` | Unless `--quiet-total` | Something went wrong and the run can not go on. Throws a panic, and the CLI exits with 1. Also added to `status.errors`. | `No data piped into replacement` |
+| `chat(msg)` | `--verbose`, also with `--quiet`, but not with `--quiet-total` | The user: what happens to each file. Only ever to stderr. Also kept in `engine.steps` of the status report. | `Open sync: a.md`, `Nothing changed in: a.md` |
+
+- What the run did is in the status report: the engine resolves with it, and `--status` prints it. So there is no extra debug output to print along the way.
+- `--debug` prints only the status report, with what went into the engine and what it did.
+- `warn()`, `error()` and `die()` add to the status report as they go, also when the quiet flags hide what they print.
 
 ## Tests
 
 - **Tests are strict, lean and focused too.** One thing per test, and the name of the test says what it is.
-- Expect the full result where it matters (`{out, err, exit}`), so nothing extra slips through.
+- Expect the full result where it matters (`{out, err, exit}` from `rr()`), so nothing extra slips through. Use `rrStatus()` for the status of the same run.
 - A known bug is a `test.todo('BUG #<issue>: ...')` until it is fixed. Then it becomes a real test.
 - A test for an issue goes in `test/js/fix/<issue>.test.js`.
 - Use Bun native file IO (`Bun.write`, `Bun.file`) and `Bun.spawnSync` in new tests (#734).

src/cli.ts

@@ -15,8 +15,7 @@ let needHelp = 0;
 if (process.argv.length < 4) {
 	if (/^(-v|--?version)$/.test(process.argv[process.argv.length - 1])) {
 		console.log(version);
-		process.exitCode = 0;
-		process.exit();
+		process.exit(0);
 	} else if (/^(-h|--?help)$/.test(process.argv[process.argv.length - 1])) {
 		needHelp = 1;
 	} else {
@@ -97,6 +96,17 @@ const yargs = require('yargs')
 	// Not .choices() as node takes the name of an encoding in any case, like `UTF8`
 	.check((argv) => Buffer.isEncoding(argv.encoding) || `Unknown encoding: ${argv.encoding}`)
 
+	.string('x')
+	.describe(
+		'x',
+		'Exclude files with a path that matches this regular expression. Will follow same regex flags and setup as the main search. Can be used multiple times.'
+	)
+	.alias('x', 'exclude-re')
+
+	.string('X')
+	.describe('X', 'Exclude files found with this glob. Can be used multiple times.')
+	.alias('X', 'exclude-glob')
+
 	.alias('E', 'engine')
 	.describe(
 		'E',
@@ -177,10 +187,6 @@ const yargs = require('yargs')
 	.alias('H', 'bail')
 	.default('H', false)
 
-	.boolean('d')
-	.describe('d', 'Print debug info')
-	.alias('d', 'debug')
-
 	.boolean('€')
 	.describe(
 		'€',
@@ -302,16 +308,6 @@ The following values defaults to \`❌\` if haystack does not originate from a f
 All variables, except from module, date objects, \`nl\` and \`_\`, has a corresponding variable name followed by \`_\` where the content has an extra space at the end (for easy concatenation). 
 `
 	)
-	.string('x')
-	.describe(
-		'x',
-		'Exclude files with a path that matches this regular expression. Will follow same regex flags and setup as the main search. Can be used multiple times.'
-	)
-	.alias('x', 'exclude-re')
-
-	.string('X')
-	.describe('X', 'Exclude files found with this glob. Can be used multiple times.')
-	.alias('X', 'exclude-glob')
 
 	/*
         .boolean('N')
@@ -395,7 +391,6 @@ N	n
 O	
 P	
 	r
-S	
 	t
 U	
 W	w
@@ -404,19 +399,56 @@ Z	z
 
 
     */
+	.boolean('S')
+	.alias('S', 'status-compact')
+	.describe(
+		'S',
+		'As the last thing before ending any execution a status report will be printed as one line of JSON. Will print even in combination with `--quiet-total`. Will incude extra info about the inner workings of the engine in combination with the `--verbose` flag.'
+	)
+
+	.boolean('status')
+	.describe(
+		'status',
+		'Same as `--status-compact`, but always starting with a `\n` followed by pretty printed JSON'
+	)
+
+	.boolean('d')
+	.describe(
+		'd',
+		'Print only the status report, with what went into the engine and what it did. Does not block the output of --output and --output-match.'
+	)
+	.alias('d', 'debug')
 
 	.help('h')
 	.describe('h', 'Display help.')
 	.alias('h', 'help')
 
 	.epilog(`Inspiration: .oO(What should 'sed' have been by now?)`);
 
+// The engine resolves with the status of the run, or throws a panic when the run can not go on.
+// The CLI is the only part that acts on the process.
+function printStatus(config, status) {
+	if (config.status) {
+		process.stdout.write('\n' + JSON.stringify(status, null, 2) + '\n');
+	} else if (config.statusCompact) {
+		process.stdout.write(JSON.stringify(status) + '\n');
+	}
+}
+
+function exitOnPanic(config, e) {
+	if (!e?.panic) {
+		throw e;
+	}
+	// A panic from the CLI itself comes before the engine has a status
+	if (e.status) {
+		printStatus(config, e.status);
+	}
+	process.exit(1);
+}
+
 function backOut(exitcode = 1) {
-	const help = yargs.showHelp();
-	const io = exitcode ? console.error : console.log;
-	//io(help);
-	process.exitCode = exitcode;
-	process.exit();
+	yargs.showHelp();
+	process.exit(exitcode);
 }
 
 const lineBreakEscapes = {'\n': '\\n', '\r': '\\r', '\u2028': '\\u2028', '\u2029': '\\u2029'};
@@ -437,111 +469,108 @@ function unescapeString(str = '') {
 	return new Function(`return '${source}'`)();
 }
 
-(function () {
-	if (0 < needHelp) {
-		return backOut(needHelp - 1);
-	}
-
-	// All options into one big config object for the rexreplace core
-	let config: any = {};
+if (0 < needHelp) {
+	backOut(needHelp - 1);
+}
 
-	// Use only camelCase full length version of settings so we make sure the core can be documented properly
-	Object.keys(yargs.argv).forEach((key) => {
-		if (1 < key.length && key.indexOf('-') < 0) {
-			config[key] = yargs.argv[key];
-		}
-	});
+// All options into one big config object for the rexreplace core
+const config: any = {};
 
-	if (config.jsPipe) {
-		config.replacementJs = config.pipe = true;
+// Use only camelCase full length version of settings so we make sure the core can be documented properly
+Object.keys(yargs.argv).forEach((key) => {
+	if (1 < key.length && key.indexOf('-') < 0) {
+		config[key] = yargs.argv[key];
 	}
+});
+
+if (config.jsPipe) {
+	config.replacementJs = config.pipe = true;
+}
 
-	outputConfig(config);
+if (config.debug) {
+	config.status = config.verbose = config.quietTotal = true;
+}
 
-	let pipeInUse = false;
-	let pipeData = '';
+outputConfig(config);
 
-	config.pipedData = null;
+config.pipedData = null;
 
-	config.showHelp = yargs.showHelp;
+config.showHelp = yargs.showHelp;
 
-	config.pattern = pattern;
+config.pattern = pattern;
 
-	config.includeGlob = yargs.argv._;
+config.includeGlob = yargs.argv._;
 
-	config.excludeGlob = [].concat(yargs.argv.excludeGlob).filter(Boolean);
+config.excludeGlob = [].concat(yargs.argv.excludeGlob).filter(Boolean);
 
-	config.excludeRe = [].concat(yargs.argv.excludeRe).filter(Boolean);
+config.excludeRe = [].concat(yargs.argv.excludeRe).filter(Boolean);
 
-	config.replacement = replacement;
+config.replacement = replacement;
 
-	// The aliases come before the escape sequences are interpreted, so `§n` gives a line break just like `\n`
-	const typed = {...config};
-	alias(config);
+// The aliases come before the escape sequences are interpreted, so `§n` gives a line break just like `\n`
+const typed = {...config};
+alias(config);
 
-	// The error shows the value as it was typed
-	const unescape = (key, name) => {
+// The error shows the value as it was typed
+const unescape = (key, name) => {
+	try {
+		config[key] = unescapeString(config[key]);
+	} catch (e) {
+		// The engine is not running yet, so the panic is caught here
 		try {
-			config[key] = unescapeString(config[key]);
-		} catch (e) {
 			die('Invalid escape sequence in ' + name + ':', typed[key]);
+		} catch (panic) {
+			exitOnPanic(config, panic);
 		}
-	};
-
-	if (!config.replacementJs) {
-		unescape('replacement', 'replacement');
 	}
+};
 
-	if (config.delimiter) {
-		unescape('delimiter', '--delimiter');
-	}
-	if (config.nameDelimiter) {
-		unescape('nameDelimiter', '--name-delimiter');
-	}
-	if (config.contentDelimiter) {
-		unescape('contentDelimiter', '--content-delimiter');
-	}
+if (!config.replacementJs) {
+	unescape('replacement', 'replacement');
+}
 
-	/*if(Boolean(process.stdout.isTTY)){
-        config.output = true;
-    }*/
+if (config.delimiter) {
+	unescape('delimiter', '--delimiter');
+}
+if (config.nameDelimiter) {
+	unescape('nameDelimiter', '--name-delimiter');
+}
+if (config.contentDelimiter) {
+	unescape('contentDelimiter', '--content-delimiter');
+}
 
-	// The engine tells how many errors there were, and the CLI turns that into the exit code
-	const run = () =>
-		rexreplace.engine(config).then((errors) => {
-			if (errors) {
+// The exit code is only set, so the process ends by itself when all output is written
+const run = () =>
+	rexreplace.engine(config).then(
+		(status) => {
+			printStatus(config, status);
+			if (status.errors.length) {
 				process.exitCode = 1;
 			}
-		});
+		},
+		(e) => exitOnPanic(config, e)
+	);
 
-	// Like grep, stdin is only read when piped data is expected. An open pipe that nobody writes to would make the run wait forever.
-	const pipeDataExpected = !config.includeGlob.length || config.replacementPipe || config.pipe;
+// Like grep, stdin is only read when piped data is expected. An open pipe that nobody writes to would make the run wait forever.
+const pipeDataExpected = !config.includeGlob.length || config.replacementPipe || config.pipe;
 
-	if (!pipeDataExpected || Boolean(process.stdin.isTTY)) {
-		return run();
-	}
+if (!pipeDataExpected || Boolean(process.stdin.isTTY)) {
+	run();
+} else {
+	// Stays null when nothing comes through the pipe
+	let pipeData = null;
 
 	process.stdin.setEncoding(config.encoding);
 
-	process.stdin.on('readable', () => {
-		let chunk = process.stdin.read();
-
-		if (null !== chunk) {
-			pipeInUse = true;
-			pipeData += chunk;
-			while ((chunk = process.stdin.read())) {
-				pipeData += chunk;
-			}
-		}
+	process.stdin.on('data', (chunk) => {
+		pipeData = (pipeData ?? '') + chunk;
 	});
 
 	process.stdin.on('end', () => {
-		if (pipeInUse) {
-			if (yargs.argv.trimPipe) {
-				pipeData = pipeData.trim();
-			}
-			config.pipedData = pipeData;
+		if (null !== pipeData && config.trimPipe) {
+			pipeData = pipeData.trim();
 		}
+		config.pipedData = pipeData;
 		run();
 	});
-})();
+}