Skip to content

Add a shared CMake helper for the backend extensions - #3111

Merged
antonwolfy merged 4 commits into
masterfrom
cmake-extension-helper
Oct 26, 2026
Merged

antonwolfy merged 4 commits into
masterfrom
cmake-extension-helper

Conversation

@antonwolfy

@antonwolfy antonwolfy commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

This PR moves the setup that the nine pybind11 extensions in dpnp/backend/extensions repeat into one CMake function, and raises the minimum CMake version so the workarounds for older versions can be removed. It is stacked on #3104 and will be rebased onto master once that PR is merged.

  1. Require CMake 3.27. cmake_minimum_required(VERSION 3.21...3.27) claimed that CMake 3.21 was enough, but pyproject.toml and the conda recipes already require CMake 3.31.6 or newer, and FetchContent_Declare(... FIND_PACKAGE_ARGS) for pybind11 needs at least 3.24. The minimum is now 3.27, which removes the 11 copies of the Windows workaround for target_link_options in CMake older than 3.27. The policy level stays at 3.27, as builds use today, so no CMake policy changes behavior.
  2. Shared helper. The nine extension CMakeLists.txt files repeated about 80 identical lines: the pybind11 module, SYCL, the _usmarray dependency, fat-binary options, include directories, math flags, device code split, coverage, the redistribution rpath and install. build_dpnp_pybind11_ext in dpnp/CMakeLists.txt now does this. Each extension lists its sources and extra include directories, keeps its own oneMKL/oneMath linking, and passes LARGE_RANGES when its kernels can be launched over more than INT_MAX work-items (indexing, statistics, ufunc, window), which adds -fno-sycl-id-queries-fit-in-int at compile and link time. A new extension can no longer miss one of these options by copying an outdated file.

The only change in the generated build is for the vm extension: it had no fat-binary block, so it was the one extension built without -fsycl-targets when DPNP_TARGET_CUDA, DPNP_TARGET_HIP or DPNP_SYCL_TARGETS is set. It now gets these options like the others. vm compiles no kernels of its own, and in CUDA/HIP builds it compiles only vm_py.cpp and the two shared elementwise helper sources that ufunc already compiles with these options.

Verification. The project was configured with CMake 4.4.4, Ninja and icpx 2026.1, with and without this change on top of the same base, and every compile and link command (ninja -t commands and compile_commands.json), the generated install rules and the configure warnings were compared:

Configuration Result
Release (oneMKL) identical
Coverage with DPNP_GENERATE_COVERAGE, DPNP_WITH_REDIST and DPNP_SYCL_TARGETS=spir64-unknown-unknown identical, except -fsycl-targets on the vm compile and link commands
Release with DPNP_USE_ONEMATH=ON identical

Windows builds are covered by CI.

  • Have you provided a meaningful PR description?
  • Have you added a test, reproducer or referred to an issue with a reproducer?
  • Have you tested your changes locally for CPU and GPU devices?
  • Have you made sure that new changes do not introduce compiler warnings?
  • Have you checked performance impact of proposed changes?
  • Have you added documentation for your changes, if necessary?
  • Have you added your changes to the changelog?

antonwolfy added a commit that referenced this pull request Oct 15, 2026
New extensions call build_dpnp_pybind11_ext instead of copying an
existing CMakeLists.txt, and request -fno-sycl-id-queries-fit-in-int
with its LARGE_RANGES option.
@antonwolfy antonwolfy added this to the 0.21.0 release milestone Oct 15, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

View rendered docs @ https://intelpython.github.io/dpnp/index.html

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Array API standard conformance tests for dpnp=0.21.0dev16=np2py314h8d9cdd5_9 ran successfully.
Passed: 1491
Failed: 0
Skipped: 6

@coveralls

coveralls commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Coverage Status

Coverage is 82.64% — cmake-extension-helper into master. No base build found for master.

@antonwolfy antonwolfy self-assigned this Oct 16, 2026
Comment thread dpnp/backend/CMakeLists.txt Outdated
Comment thread CMakeLists.txt
@antonwolfy
antonwolfy force-pushed the cmake-extension-helper branch from 4edef44 to 7e724b0 Compare October 22, 2026 19:26
Base automatically changed from fix-id-queries-link-flag to master October 23, 2026 11:08
All build requirements (pyproject.toml and the conda recipes) already need
CMake 3.31.6 or newer, and FetchContent with FIND_PACKAGE_ARGS needs 3.24,
so the minimum of 3.21 was not accurate. Raising it to 3.27 removes the
need for the Windows workaround for target_link_options in older CMake
versions. The policy level stays at 3.27, as before.
The nine extensions in dpnp/backend/extensions repeated the same target
setup. Move it into build_dpnp_pybind11_ext in dpnp/CMakeLists.txt; each
extension now lists its sources and extra include directories, keeps its
own oneMKL/oneMath linking, and passes LARGE_RANGES when its kernels can
be launched over more than INT_MAX work-items.

The vm extension now also gets the -fsycl-targets options when building
for custom SYCL targets, like the other extensions.
@antonwolfy
antonwolfy force-pushed the cmake-extension-helper branch from 7e724b0 to ad09ed0 Compare October 23, 2026 11:13
@vlad-perevezentsev
vlad-perevezentsev self-requested a review October 26, 2026 10:36

@vlad-perevezentsev vlad-perevezentsev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Thank you @antonwolfy

@antonwolfy
antonwolfy merged commit 2fe6540 into master Oct 26, 2026
108 of 110 checks passed
@antonwolfy
antonwolfy deleted the cmake-extension-helper branch October 26, 2026 14:33
github-actions Bot added a commit that referenced this pull request Oct 26, 2026
This PR moves the setup that the nine pybind11 extensions in
`dpnp/backend/extensions` repeat into one CMake function, and raises the
minimum CMake version so the workarounds for older versions can be
removed. It is stacked on #3104 and will be rebased onto master once
that PR is merged.

1. **Require CMake 3.27.** `cmake_minimum_required(VERSION 3.21...3.27)`
claimed that CMake 3.21 was enough, but `pyproject.toml` and the conda
recipes already require CMake 3.31.6 or newer, and
`FetchContent_Declare(... FIND_PACKAGE_ARGS)` for pybind11 needs at
least 3.24. The minimum is now 3.27, which removes the 11 copies of the
Windows workaround for `target_link_options` in CMake older than 3.27.
The policy level stays at 3.27, as builds use today, so no CMake policy
changes behavior.
2. **Shared helper.** The nine extension `CMakeLists.txt` files repeated
about 80 identical lines: the pybind11 module, SYCL, the `_usmarray`
dependency, fat-binary options, include directories, math flags, device
code split, coverage, the redistribution rpath and install.
`build_dpnp_pybind11_ext` in `dpnp/CMakeLists.txt` now does this. Each
extension lists its sources and extra include directories, keeps its own
oneMKL/oneMath linking, and passes `LARGE_RANGES` when its kernels can
be launched over more than `INT_MAX` work-items (`indexing`,
`statistics`, `ufunc`, `window`), which adds
`-fno-sycl-id-queries-fit-in-int` at compile and link time. A new
extension can no longer miss one of these options by copying an outdated
file.

The only change in the generated build is for the `vm` extension: it had
no fat-binary block, so it was the one extension built without
`-fsycl-targets` when `DPNP_TARGET_CUDA`, `DPNP_TARGET_HIP` or
`DPNP_SYCL_TARGETS` is set. It now gets these options like the others.
`vm` compiles no kernels of its own, and in CUDA/HIP builds it compiles
only `vm_py.cpp` and the two shared elementwise helper sources that
`ufunc` already compiles with these options.

**Verification.** The project was configured with CMake 4.4.4, Ninja and
icpx 2026.1, with and without this change on top of the same base, and
every compile and link command (`ninja -t commands` and
`compile_commands.json`), the generated install rules and the configure
warnings were compared:

| Configuration | Result |
|---|---|
| Release (oneMKL) | identical |
| Coverage with `DPNP_GENERATE_COVERAGE`, `DPNP_WITH_REDIST` and
`DPNP_SYCL_TARGETS=spir64-unknown-unknown` | identical, except
`-fsycl-targets` on the `vm` compile and link commands |
| Release with `DPNP_USE_ONEMATH=ON` | identical |

Windows builds are covered by CI. 2fe6540
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants