Build and test conda-recipe-cf for Intel Python CI

.github/workflows/intel-ci.yml

@@ -0,0 +1,135 @@
+# Intel Python CI (gh-ci) for mkl_random: builds the conda-forge recipe, conda-recipe-cf, for every
+# platform and Python version it declares, as ci/matrix.py reads them, and tests each package in
+# a clean conda-forge environment with ci/test.py, exactly as Jenkins does. A pull request builds
+# only the oldest and the newest regular Python versions. Intel's Jenkins waits for this whole
+# run, then tests and scans each package on its own machines; see ci/README.md. Steps run their
+# commands as Python, so they are the same on Linux and Windows.
+name: Intel Python CI
+
+on:
+  push:
+    branches: [master]
+    tags: ['*']
+  pull_request:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: intel-ci-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  matrix:
+    name: Platforms and Python versions
+    runs-on: ${{ vars.GH_CI_RUNNER || 'ubuntu-latest' }}
+    timeout-minutes: 10
+    outputs:
+      matrix: ${{ steps.declared.outputs.matrix }}
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+
+      - name: Read the declared platforms and Python versions
+        id: declared
+        run: python ci/matrix.py --event "$GITHUB_EVENT_NAME" --github-output matrix
+
+  build:
+    name: ${{ matrix.name }} Python ${{ matrix.python }}
+    needs: matrix
+    runs-on: ${{ matrix.platform == 'windows' && (vars.GH_CI_WINDOWS_RUNNER || 'windows-2022') || (vars.GH_CI_RUNNER || 'ubuntu-latest') }}
+    timeout-minutes: 90
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJSON(needs.matrix.outputs.matrix) }}
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+          # The recipe takes its version from git describe.
+          fetch-depth: 0
+
+      # Miniforge 26.9.4-0 fails setup-miniconda's conda init on Windows runners; 26.7.2-0 works.
+      - uses: conda-incubator/setup-miniconda@be893c923ea9cf1cf7cd510fbdde27c7e18cbdcb # v4.1.0
+        with:
+          miniforge-version: 26.7.2-0
+          activate-environment: ""
+          conda-remove-defaults: "true"
+
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+
+      # ci/build.py runs with the conda base install's Python, from $CONDA, and conda-build runs
+      # as a module of it.
+      - name: Build and run the recipe tests
+        shell: python
+        env:
+          PYTHON_VERSION: ${{ matrix.python }}
+        run: |
+          import os, subprocess, sys
+          base = os.environ['CONDA']
+          python = os.path.join(base, 'python.exe' if os.name == 'nt' else os.path.join('bin', 'python'))
+          subprocess.run([python, '-m', 'conda', 'install', '-y', '-q', '-n', 'base',
+                          '--override-channels', '-c', 'conda-forge', 'conda-build'], check=True)
+          output = os.path.join(os.environ['RUNNER_TEMP'], 'output')
+          sys.exit(subprocess.run([python, 'ci/build.py', '--python', os.environ['PYTHON_VERSION'],
+                                   '--output', output]).returncode)
+
+      - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
+        with:
+          name: mkl_random ${{ matrix.name }} Python ${{ matrix.python }}
+          path: ${{ runner.temp }}/output/*.conda
+          if-no-files-found: error
+          # Jenkins downloads them within minutes of the run, so pull requests and branches keep
+          # them 3 days; tags keep 30.
+          retention-days: ${{ startsWith(github.ref, 'refs/tags/') && 30 || 3 }}
+
+  test:
+    name: Test ${{ matrix.name }} Python ${{ matrix.python }}
+    needs: [matrix, build]
+    runs-on: ${{ matrix.platform == 'windows' && (vars.GH_CI_WINDOWS_RUNNER || 'windows-2022') || (vars.GH_CI_RUNNER || 'ubuntu-latest') }}
+    timeout-minutes: 45
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJSON(needs.matrix.outputs.matrix) }}
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+
+      - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
+        with:
+          name: mkl_random ${{ matrix.name }} Python ${{ matrix.python }}
+          path: ${{ runner.temp }}/artifacts
+
+      - uses: conda-incubator/setup-miniconda@be893c923ea9cf1cf7cd510fbdde27c7e18cbdcb # v4.1.0
+        with:
+          miniforge-version: 26.7.2-0
+          activate-environment: ""
+          conda-remove-defaults: "true"
+
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+
+      # ci/hooks.py runs itself again with the conda base install's Python, from $CONDA.
+      - name: Install the package and run ci/test.py
+        shell: python
+        env:
+          PYTHON_VERSION: ${{ matrix.python }}
+          EVENT: ${{ startsWith(github.ref, 'refs/tags/') && 'tag' || github.event_name == 'pull_request' && 'pr' || 'push' }}
+          COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
+        run: |
+          import os, subprocess, sys
+          env = os.environ
+          temp = env['RUNNER_TEMP']
+          sys.exit(subprocess.run([sys.executable, 'ci/hooks.py', '--python', env['PYTHON_VERSION'],
+                                   '--artifacts', os.path.join(temp, 'artifacts'),
+                                   '--work', os.path.join(temp, 'hooks'),
+                                   '--event', env['EVENT'], '--commit', env['COMMIT']]).returncode)

ci/README.md

@@ -0,0 +1,28 @@
+# How Intel Python CI tests mkl_random
+
+Intel Python CI (gh-ci) builds and tests mkl_random's conda-forge packages in GitHub Actions. Intel's Jenkins then tests each package again on Intel machines and scans it, and on a release tag publishes the packages it tested. Everything it needs from this repository is in this directory, one workflow, and the recipe it builds.
+
+| File | Role |
+|---|---|
+| `../conda-recipe-cf/` | The recipe gh-ci builds. `conda_build_config.yaml` lists the Python versions. The recipe builds on Linux and Windows unless a platform-only skip in `meta.yaml`, such as `skip: true  # [win]`, rules one out. |
+| `../.github/workflows/intel-ci.yml` | One job reads the matrix, one job per platform and Python version builds and uploads `mkl_random <Platform> Python <version>`, and one job per package tests it. |
+| `matrix.py` | Reads the platforms and Python versions from the recipe. A pull request gets the oldest and the newest regular versions. |
+| `build.py` | Builds one Python version with conda-build, which runs the recipe's own tests. |
+| `hooks.py` | The workflow's test jobs: installs one package into a clean conda-forge environment with the requirements in `config.yml`, and runs `test.py` the way Jenkins does. |
+| `config.yml` | Tells gh-ci which workflow builds the packages, what its artifacts are called, and what the tests need. |
+| `test.py` | Runs the installed package's test suite with pytest, in Actions and again in Jenkins. |
+
+The scripts need only the Python standard library and are the same in every repository gh-ci tests, except that `matrix.py` here looks for `conda-recipe-cf/` first. Scanning is not the repository's to define: Jenkins runs its own scans.
+
+**A pull request into `master`.** Actions builds the oldest and the newest regular Python versions on Linux and Windows and tests each package. Jenkins notices the pull request within 3 minutes, waits for that whole run, and then tests and scans every package at once. Each cell posts its own status on the pull request, and `intel-ci/pr` sums them up.
+
+**A release tag.** Actions builds every declared version, and Jenkins tests and scans all of them before it publishes the release.
+
+To support another Python, add it to `conda-recipe-cf/conda_build_config.yaml`; nothing else lists versions. Try it locally from a conda base environment with conda-build:
+
+```bash
+python ci/matrix.py                           # every platform and Python version
+python ci/matrix.py --event pull_request      # what a pull request builds
+python ci/build.py --python VERSION --output /tmp/out     # VERSION: one matrix.py lists
+python ci/hooks.py --python VERSION --artifacts /tmp/out --work /tmp/hooks
+```