Skip to content

Limit how many files are handled at the same time to close #708 - #740

Merged
mathiasrw merged 3 commits into
mainfrom
fix/708
Oct 21, 2026
Merged

mathiasrw merged 3 commits into
mainfrom
fix/708

Conversation

@mathiasrw

@mathiasrw mathiasrw commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

Closes #708

Change

  • A pool of workers handles the files in the default flow. Each worker takes the next file when it is done with one.
  • The number of workers is 16, or two per CPU core if that is more (os.availableParallelism() * 2). With fewer files than workers, all files are handled at once like before.
  • One file is handled in the synchronous flow, like with --void-async.
  • --void-async, -o and -m use the same pool with one worker, so the output keeps the order of the file list.
  • A file that can not be read gave a stack trace in the synchronous flow. It now gives an error and the run goes on, like in the async flow. This was needed as one file now uses the synchronous flow.

Good to know

  • With the open files limited to 256 (ulimit -Hn 256) and 3000 files on Node: main changes 245 files and gives 2755 EMFILE errors. This branch changes all 3000.

@mathiasrw
mathiasrw merged commit 99d4caa into main Oct 21, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Limit how many files are handled at the same time

1 participant