Skip to content

Fix intermittent Test_Powders results (header parsing) and other undefined behaviour - #2770

Merged
willend merged 2 commits into
mainfrom
runtime-ub-fixes
Oct 7, 2026
Merged

willend merged 2 commits into
mainfrom
runtime-ub-fixes

Conversation

@willend

@willend willend commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Free-form text area

Please describe what your PR is adding in terms of features or bugfixes:

Fix intermittent Test_Powders results (header parsing) and other undefined behaviour

Branch runtime-ub-fixes, from main. Runtime and component files only, no code-generator changes. Two commits:

  1. 6b2138b fixes header-key matching, plus smaller fixes for undefined behaviour.
  2. da367a6 fixes reflectivity-lib (optional header keys, KINEMATIC indices).

Root cause of the intermittent Test_Powders results

  • Table_ParseHeader() (read_table-lib) used strcasestr(header, key), which matches the first occurrence of the key anywhere in the header, including inside other words.
  • PowderN and Single_crystal convert CIF files with cif2hkl into a tmpnam() file. cif2hkl writes its command line, including that random file name (e.g. /var/tmp/tmp.0.aVcX9q), into the header above the real # Vc 181.085 line.
  • When the random part contains "vc" (any case), that rank gets V_0 = 0 and absorbs every particle: one MPI rank's contribution goes missing (e.g. 2.94e8 instead of 3.92e8 with 4 ranks).
  • Similarly, "dw" in the name sets a bogus Debye-Waller factor: same events, different weights.
  • Each rank draws its own temp names, two per rank in Test_Powders, so failures get more frequent with more ranks. The MPI code itself is fine; MPI just makes this bug show up more often.

Fixes

  • read_table-lib: Table_ParseHeader() matches keys as whole words, with word boundaries only where the key itself starts or ends with [A-Za-z0-9_]. Keys like "sigma_a " or "e_min=" match as before.
  • reflectivity-lib (McXtrace):
    • COATING files: d is optional and defaults to 0, instead of strtod(NULL). It used to be found by accident in "density" (also giving 0), so all McXtrace mirror/coating instruments crashed with whole-word matching until this fix.
    • Type detection: no strtol(NULL) when a file has no Z.
    • KINEMATIC files: the check looked for a 5th key that is never parsed, so it always failed, and the values were read from shifted indices. Both fixed.
  • PowderN, Isotropic_Sqw (McStas and McXtrace) and Powder_process (McStas): the line list is calloc'd, because the multiplicity check can read one entry ahead into uninitialised memory.
  • Pol_guide_vmirror: copies at most 6 values into its r..ParToFunc[6] arrays. A longer vector used to overflow (flagged by UBSan in ILL_H5).
  • FUNNEL, sort_absorb_last(): *multiplier is now also set on the serial path, and there's no division by zero when every particle is absorbed.

Not changed

  • tmpnam() is kept: MSVC has no mkstemp(), and with the parser fix the random names are harmless.

Testing (macOS arm64)

  • Test_Powders: 40 of 40 MPI runs identical (4 ranks, fixed seed). Before: about 1 in 10–20 runs differed.
  • Header-parsing unit test: temp names containing "Vc"/"dw7": old Vc=0 / DW=7, new correct. Across all 343 shipped data files, the neutron powder/S(q,w)/crystal headers parse identically.
  • All 444 McStas and McXtrace examples (default parameters, 1e4 rays, seed 1000): identical detector output to main, and no new failures.
  • ILL_H5 under ASan/UBSan: clean (before: out-of-bounds writes in Pol_guide_vmirror).
  • Linux and Windows/MSVC are untested.

Related

  • The wrong values in ILL_H5's {...} vectors with expressions are a code-generator bug, fixed in Cogen improve comments review #2767. This branch only stops the overflow they caused.
  • This may make suppress-cif2hkl-Test_Powders / just-suppress-cif2hkl-Test_Powders unnecessary.

Declaration of use of AI-tools

  • Please add a checkmark here if you used AI-tools during the work for this contribution
  • Furter, please describe how / where and for what the tools were used:

^ was 🤖 Generated with Claude Code and using /ponytail full


Development OS / boundary conditions

Please describe what OS you developed and tested your additions on, and if any special dependencies are required:


PR Checklist for contributing to McStas/McXtrace

For a coherent and useful contribution to McStas/McXtrace, please fill in relevant parts of the checklist:

  • My contribution contains something else

    • Explanation is added in free form text above or below the checklist

willend and others added 2 commits October 7, 2026 14:22
- read_table-lib: Table_ParseHeader() now matches symbols as whole words
  (word boundaries where the symbol starts/ends with [A-Za-z0-9_]).
  Before, "Vc" or "DW" could match inside the random tmpnam() file name
  that cif2hkl writes into its header above the real values. That gave
  V_0=0 (all particles absorbed) or a bogus Debye-Waller factor on
  whichever MPI rank drew such a name: the intermittent Test_Powders
  results. Neutron powder/Sqw/crystal headers of all shipped data files
  parse unchanged.
- PowderN, Isotropic_Sqw (McStas/McXtrace), Powder_process (McStas):
  calloc the line list; the multiplicity check may read one entry ahead.
- Pol_guide_vmirror: copy at most 6 values into r..ParToFunc[6].
- sort_absorb_last (FUNNEL): set *multiplier also on the serial path,
  and do not divide by zero when every particle is absorbed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- COATING: "d" is optional (no shipped coating file has it); default to 0
  instead of strtod(NULL). It used to be found by accident, as the "d" in
  "density", which also gave 0. With whole-word header matching this
  crashed all McXtrace mirror/coating instruments.
- COATING type guess: do not strtol(NULL) when a file has no "Z".
- KINEMATIC: four keys are parsed, so check indices 0-3 (it checked a
  non-existent 5th one and always failed) and read gamma/lambda/rho_ab
  from 1-3 instead of 2-4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@willend

willend commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Test results (macOS arm64, conda-forge mccode-dev env)

Test Before (main) After (runtime-ub-fixes)
Test_Powders, comp=1, 4 MPI ranks, 1e5 rays, fixed seed about 1 in 10–20 runs wrong: one rank with V_0=0 gives Sph_mon_I=2.94e8 instead of 3.92e8, or a wrong DW changes I but not N 40 of 40 runs identical, Sph_mon_I=3.92196e+08
Header parsing, cif2hkl output named tmp.0.aVcX9q Vc=0 Vc=181.085
Header parsing, cif2hkl output named tmp.0.k2dw7Q DW=7 (bogus) DW not found (correct)
Header parsing, normal files (Al.laz, Rb_liq_coh.sqw) Vc=66.4 / sigma_abs=0.38 identical
Neutron powder/S(q,w)/crystal header keys, all 343 shipped data files – parsed values identical
ILL_H5 under UBSan 3 out-of-bounds writes in Pol_guide_vmirror (index 6, double[6]) clean
McXtrace mirror/coating instruments (17), with whole-word matching only crash (strtod(NULL) in reflectivity-lib) fixed by da367a6fa: results identical to main
All 444 McStas and McXtrace examples (default parameters, 1e4 rays, seed 1000) baseline 444 of 444 identical detector output, no new failures

How it was tested:

  • Before and after: used the same installed generator. Branch A's runtime and component files were put first on the -I search path.
  • mcrun -I: keeps only the last -I, so one combined search directory was used.
  • Sanitizers: Apple /usr/bin/clang. The conda clang's ASan runtime hangs at startup on this macOS.
  • Not tested: Linux, Windows/MSVC.

@willend

willend commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

https://github.com/mccode-dev/McCode/actions/runs/37625771546/job/112807005872?pr=2770#step:34:391 is hanging on an issue that should be fixed with #2767 in place (correction of {} parsing). Merge.

@willend
willend merged commit 5160176 into main Oct 7, 2026
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant