Skip to content

FIX Only skip the "sequential_blas_under_openmp" limit for OpenBLAS with OpenMP - #283

Merged
ogrisel merged 2 commits into
joblib:masterfrom
cakedev0:fix-sequential-blas-select
Oct 28, 2026
Merged

ogrisel merged 2 commits into
joblib:masterfrom
cakedev0:fix-sequential-blas-select

Conversation

@cakedev0

@cakedev0 cakedev0 commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

limits="sequential_blas_under_openmp" doesn't limit anything as soon as OpenBLAS is loaded, whatever its threading layer. select() keeps the libraries matching at least one of its kwargs (as documented), so select(internal_api="openblas", threading_layer="openmp") also matches pthreads OpenBLAS, i.e. the one of the numpy and scipy wheels. It also matches any library with the OpenMP threading layer (e.g. BLIS).

So when I switched to it in scikit-learn/scikit-learn#35180 KMeans' Lloyd iterations got ~2.8x slower on main with 2 cores (29 ms -> 81 ms per iteration, 200k x 15 float64, 150 clusters, Xeon 6787P, PyPI numpy/scipy). The per-chunk GEMMs called from the OpenMP loop are multi-threaded again.
OPENBLAS_NUM_THREADS=1 brings it back to 29 ms.

The test computed its expected value with the same select() call, so it couldn't catch this. It now checks the library info directly: on upstream master it fails with pthreads OpenBLAS, and it passes with this fix.

cc @itamarst @ogrisel

cakedev0 and others added 2 commits October 28, 2026 15:03

@ogrisel ogrisel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch indeed...

@ogrisel
ogrisel merged commit e36e58f into joblib:master Oct 28, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants