Read the piped paths of --glob-pipe as UTF-8 #790

Today

With --glob-pipe, the piped file names are decoded with --encoding. So with -e latin1, a file name with letters that are not ASCII is turned into another name, finds no file, and is left out without a word.

> printf foo > æ.file; printf foo > b.file
> printf 'æ.file\nb.file\n' | rexreplace foo bar -g -e latin1
b.file
> echo $?
0
> cat æ.file
foo

Without -e latin1, æ.file is changed as it should be. Seen on main (6ea7b83) with Node 22 and Bun.

The cause is process.stdin.setEncoding(config.encoding) in src/cli.ts:587. That is right when the piped data is content, but with --glob-pipe it is a list of paths, and paths are not in the encoding of the files.

The README tells users to use --encoding latin1 to run binary safe, and --glob-pipe --null-in for paths from find -print0. So the two are likely to be used together, and then files with such names are missed.

Change

With --glob-pipe, decode the piped data as UTF-8, whatever --encoding is.

Add a test with a file name that is not ASCII, piped with -g -e latin1.

Good to know

A file name that is not valid UTF-8 can still not be piped in this way. That is a separate, much rarer case.

Reference

• Add --glob-pipe to pipe in the files to work on to close #711 #774 / Add --glob-pipe to pipe in the files to work on #711