Skip to content

Fix NaNs in test_discrete_gradient on DOLFINx nightly - #268

Merged
finsberg merged 1 commit into
mainfrom
finsberg/fix-discrete-gradient-nightly
Oct 12, 2026
Merged

finsberg merged 1 commit into
mainfrom
finsberg/fix-discrete-gradient-nightly

Conversation

@finsberg

@finsberg finsberg commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes the three test_discrete_gradient[*-False-tetrahedron] failures ("nan location mismatch") on the DOLFINx nightly leg. The bug was in the test, not in scifem.interpolation.interpolation_matrix.

The MatrixCSR branch of the test built one input vector on the column index map of DOLFINx's discrete_gradient, and passed it to the mult of both matrices. MatrixCSR.mult indexes the vector with the matrix's own column numbering, so the vector must be on that matrix's column map. On nightly the two maps differ:

  • dolfinx.fem.discrete_gradient uses V's map as is (e.g. 8 ghosts).
  • dolfinx.fem.create_matrix, which interpolation_matrix uses, adds ghosts in a different order (9–10 ghosts; at degree 3, 117 vs 81).

So scifem's matrix read past the end of the vector: column index 241 into a vector of length 206 on one rank. Those columns have zero coefficients, so the product was usually right. It became NaN whenever the memory read happened to hold one, which is why the failures were intermittent and only appeared under MPI.

The test now builds u on each matrix's own column map.

Testing

Locally on DOLFINx nightly, against scifem main:

  • test_discrete_gradient 5 times each at mpirun -n 3 and -n 4: the old test failed in 9 of 10 runs, the fixed test passed all 10.
  • tests/test_interpolation.py passes serially and at 4 ranks.
  • pre-commit passes.

🤖 Generated with Claude Code

The MatrixCSR branch of test_discrete_gradient built one input vector on
the column index map of DOLFINx's discrete gradient and passed it to the
mult of both matrices. MatrixCSR.mult indexes the vector with the
matrix's own column numbering, so the vector has to be on that matrix's
column map. On DOLFINx nightly the two maps differ: discrete_gradient
uses V's map as is, while create_matrix, which interpolation_matrix
uses, adds ghosts in another order. scifem's matrix then read past the
end of the vector. Those columns have zero coefficients, so the product
was usually right, and NaN when the memory read held one, which is why
the failures came and went, and only in parallel.

Build `u` on each matrix's column map instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@finsberg finsberg changed the title Give each matrix its own input vector in test_discrete_gradient Fix NaNs in test_discrete_gradient on DOLFINx nightly Oct 12, 2026
@finsberg
finsberg merged commit f24a9fb into main Oct 12, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant