Repository navigation
Faster CI: batched test compilation, sanitizer builds in their own jobs - #238
Merged
Merged
Conversation
Single-file tests are compiled six to a translation unit, each file in its own namespace, so shared templates are instantiated once per batch instead of once per file. Tests with C entry points or their own flags stay standalone. NUMSIM_MATERIALS_BATCH_TESTS=OFF restores one executable per test. CI runs plain and ASan+UBSan builds of each compiler as separate jobs.
Member
Author
|
First CI run:
Wall clock 16 min, down from 45–55 min. 751 + 735 = 1486: the same tests as before. The plain jobs also run the report tests. No memory problems with batch size 6. |
This was referenced Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI took 45–55 minutes per PR, almost all of it compiling: the tests themselves run in about 30 s. This keeps ASan+UBSan on every PR for both compilers, and makes it cheaper.
Where the time went
numsim_add_testbuilds each of the ~78 test files twice: plain, and an ASan+UBSan copy. The sanitizer copy compiles about 1.9× slower.-ftime-report: 66 % of the time is code generation for instantiated templates (callgraph functions expansionalone is 48 %). Every test file instantiates the same engine, tmech and gtest templates again.Change
Batched tests.
add_numsim_testcollects the single-file tests. Six at a time go into one generated translation unit (tests/batch_tests.cmake):#includeof the batch is hoisted above the namespaces, so the inner ones are no-ops;add_numsim_test(...)lines stay as they are, so open PRs that add tests merge without conflicts.Built standalone, as before:
test_read_check_layout);test_material_conformance,test_host_neutral_includes);NUMSIM_MATERIALS_DEFINE_*orextern "C"), whose symbols ignore namespaces. This is detected from the source.-DNUMSIM_MATERIALS_BATCH_TESTS=OFFgives one executable per test again, for TDD on one file.NUMSIM_MATERIALS_TEST_BATCH_SIZEsets the batch size (default 6).CI.
{g++-14, clang++-19} × {plain, asan}: four parallel jobs.numsim_materials_asan_tests) and runctest -R _asan$.NUMSIM_SANITIZER_TESTS=OFFand also run the report tests.-j$(nproc).Checks
test_mean_field_local) takes 3.6 GB. The runners have 16 GB for 4 parallel compiles. If CI shows memory pressure, the batch size goes down.Merge order
Independent of the epic stack. #228 also edits
ci.yml: whichever merges second needs a small conflict resolution, which I will do.