Skip to content

ci: reorganize ARM testlists by family and add chip build filter - #20787

Open
Swatantra-66 wants to merge 1 commit into
apache:masterfrom
Swatantra-66:ci/arm-family-build-filter
Open

Swatantra-66 wants to merge 1 commit into
apache:masterfrom
Swatantra-66:ci/arm-family-build-filter

Conversation

@Swatantra-66

Copy link
Copy Markdown
Contributor

Summary

Implements Phase 2 of #20483 ("Improve and optimize NuttX CI").

Background & Problem

  • Previously, ARM testlists (arm-01 through arm-14) partitioned 1,230 defconfigs alphanumerically, resulting in significant compile duration imbalance (e.g. arm-04 and arm-10 built only ~41 defconfigs while arm-13 built 199), causing faster runners to sit idle waiting for the slowest container.
  • Any PR touching an ARM chip triggered all 14 containers regardless of whether changes were isolated to a specific vendor/chip family.

Key Changes

  1. Balanced Family Testlists (tools/ci/testlist/):

    • Replaced arm-01..14 with 13 balanced vendor/family-based testlists (~60 to 120 defconfigs each):
      • arm-imxrt.dat: i.MX RT and i.MX6/9/8 (76 defconfigs)
      • arm-nrf.dat: nRF52, nRF53, nRF54L, nRF91 (89 defconfigs)
      • arm-nxp.dat: LPC, Kinetis, S32K, LC823450 (114 defconfigs)
      • arm-rp2040.dat: RP2040 and RP2350 (130 defconfigs)
      • arm-sam.dat: SAM34, SAMA5, SAMD/E/V (88 defconfigs)
      • arm-ti-cxd56.dat: Tiva, RM57, TMS570, CXD56xx (73 defconfigs)
      • arm-stm32f4-1.dat & arm-stm32f4-2.dat: STM32F4 (split 50/50: 65 & 77 defconfigs)
      • arm-stm32f0-f3.dat: STM32F0, F1, F2, F3 (109 defconfigs)
      • arm-stm32h7.dat: STM32H7 (97 defconfigs)
      • arm-stm32l-u-g.dat: STM32L, U, G, WB, WL (80 defconfigs)
      • arm-stm32-other.dat: STM32F7, H5, C0, C5, N6 (53 defconfigs)
      • arm-other.dat: Niche/generic ARM chips (GD32, AT32, MPS, Max326, RTL872x, etc.)
    • Preserves 100% parity across all 1,230 defconfigs, 209 CMake builds, and 14 blacklist entries (0 missing).
  2. Chip-Isolated Build Filtering (.github/workflows/arch.yml):

    • In get-arch, if a PR touches only a single ARM family (under arch/arm/src/<chip>/ and/or boards/arm/<chip>/), CI sets arm_family to compile only that family's container (e.g. PR on i.MX RT only runs arm-imxrt).
    • Safely falls back to full ARM regression (all 13 containers) if:
      • Common/core files (arch/arm/src/common, arch/arm/src/stm32/, sched/, drivers/, etc.) are touched
      • Multiple chip families are modified
      • A diff exceeds 100 files, GitHub CLI is unavailable, or changed-file list is incomplete / count is unknown
  3. Workflow Integration (.github/workflows/build.yml):

    • Updated Linux-Arch input array to match the 13 family targets.
    • Updated reusable workflow calls to local ./.github/workflows/arch.yml.

Fixes #20483.

Impact

  • Is new feature added?: Yes (CI optimization)
  • Impact on user: None.
  • Impact on build: Significantly reduces CI runner time for chip-isolated PRs (saving 11–12 containers per PR) and balances runner workload across jobs.
  • Impact on hardware: None.
  • Impact on documentation: None.
  • Impact on security: None.
  • Impact on compatibility: 100% backward compatible. Full regression suite is retained.

Testing

I confirm that changes are verified on local setup:

  • Build Host: Windows 11 / Linux, Python 3.10+
  • Target(s): CI Workflows (.github/workflows/) and Testlists (tools/ci/testlist/)

Verification Details:

  1. Validated YAML syntax with yaml.safe_load across both modified workflow files (arch.yml, build.yml).
  2. Verified Python family resolution with test cases:
    • Single family (arch/arm/src/imxrt/... -> arm-imxrt)
    • Multi-family / common files (arch/arm/src/common/... -> all 13 containers)
    • Diff truncation / count unknown / large PR edge cases -> safe fallback to all 13 containers
  3. Audited defconfig resolution across all 1,230 ARM defconfigs:
    • 0 missing defconfigs compared to master
    • All 209 CMake configurations preserved
    • All 14 blacklist entries preserved
  4. Confirmed zero CRLF line endings (pure Unix LF) across all modified and new files.
  5. GitHub Actions CI will automatically validate execution upon PR creation.

@github-actions github-actions Bot added Area: CI Size: M The size of the change in this PR is medium labels Oct 18, 2026
@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

MemBrowse Memory Report · Explore what changed →

No memory changes detected for:


Catch memory regressions in CI with MemBrowse. Free for open source.

@Swatantra-66
Swatantra-66 force-pushed the ci/arm-family-build-filter branch from 2d8e4dd to f76953a Compare October 19, 2026 08:31
Comment thread tools/ci/testlist/arm-imx.dat
Comment thread tools/ci/testlist/arm-nrf.dat Outdated
Comment thread tools/ci/testlist/arm-rp2xxx.dat
Comment thread tools/ci/testlist/arm-stm32f0-f3.dat
Comment thread tools/ci/testlist/arm-ti.dat
@Swatantra-66
Swatantra-66 force-pushed the ci/arm-family-build-filter branch from f76953a to 4843182 Compare October 19, 2026 18:02
@github-actions github-actions Bot added Size: L The size of the change in this PR is large and removed Size: M The size of the change in this PR is medium labels Oct 19, 2026

@raiden00pl raiden00pl left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this done with AI? Blindly modifying CI with AI is not a very good idea.

Comment thread .github/workflows/arch.yml Outdated
Comment thread .github/workflows/arch.yml
Comment thread .github/workflows/build.yml
Comment thread tools/ci/testlist/arm-stm32l-u-g.dat Outdated
Reorganize ARM CI testlists from arbitrary alphanumeric partitions
(arm-01..14) into 13 balanced vendor/family testlists, and add
chip-isolated build filtering in CI (Phase 2 of apache#20483).

Summary of changes:
- tools/ci/testlist/: Replace arm-01..14 with 13 balanced family
  testlists (arm-imx, arm-nrf, arm-nxp, arm-rp2xxx, arm-sam, arm-ti,
  arm-stm32f4-1/2, arm-stm32f0-f3, arm-stm32h7, arm-stm32l-u-g,
  arm-stm32-other, arm-other). Full parity preserved for all defconfigs,
  CMake builds, and blacklist entries with alphabetical sorting across all files.
- .github/workflows/arch.yml: Detect isolated single-family ARM PRs
  and build only the matching container(s). Automatically fall back
  to all ARM containers if common files, unknown chips, multiple
  families, or truncated diffs are detected.
- .github/workflows/build.yml: Update Linux-Arch board matrix to the
  13 family targets and use local workflow references.

Resolves: apache#20483
Signed-off-by: Swatantra Yadav <maverickswatantra@gmail.com>
@Swatantra-66
Swatantra-66 force-pushed the ci/arm-family-build-filter branch from aa6d87f to 2f5bc92 Compare October 20, 2026 09:08
@Swatantra-66

Copy link
Copy Markdown
Contributor Author

Hi @raiden00pl, thank you for the review All points have been addressed:

  1. stm32f0l0g0 / stm32: Removed from common_dirs. That was an obsolete reference from prior to the !arch/stm32: split stm32 and stm32f0l0g0 into per-chip-family directories #19143 directory split.
  2. common/stm32 targeting: Implemented! Any change touching arch/arm/src/common/stm32/ or boards/arm/common/stm32/ (or cross-STM32 families) now sets the family to arm-stm32. This builds all 6 STM32 containers (arm-stm32*) while skipping all 7 non-STM32 containers (arm-nxp, arm-nrf, arm-sam, arm-imx, arm-ti, arm-rp2xxx, arm-other).
  3. uses: ./.github/workflows/arch.yml: In GitHub Actions, referencing @master downloads the workflow from apache/nuttx:master, which ignores PR changes to arch.yml during CI testing. Using ./.github/workflows/arch.yml allows GitHub Actions to test the updated arch.yml from this PR branch.
  4. Toolchain / Compiler Parity: The toolchain flags have been completely restored to 1:1 exact parity with master (arm-01..14.dat). Verified across all 13 testlists:
    Defconfigs: 1,226 in master, 1,226 in PR (0 additions, 0 deletions)
    CMake targets: 209 in master, 209 in PR (0 additions, 0 deletions)
    Blacklist: 14 in master, 14 in PR (0 additions, 0 deletions)

@raiden00pl

Copy link
Copy Markdown
Member

this PR will also break the CI for the apps, so we need a PR on the apps side that is ready to be merged after this changes

@acassis

acassis commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

this PR will also break the CI for the apps, so we need a PR on the apps side that is ready to be merged after this changes

@Swatantra-66 maybe you should already prepare the PR for nuttx-apps, to avoid breaking the CI for a long time

@Swatantra-66

Copy link
Copy Markdown
Contributor Author

Hi @raiden00pl @acassis,

I have submitted the companion PR on the apps side: apache/nuttx-apps#3891.

It updates .github/workflows/build.yml to the 13 family testlists and includes Depends-On: apache/nuttx#20787 so its CI will build and test directly against this branch. Once this PR merges, apache/nuttx-apps#3891 can be merged immediately to keep the CI completely uninterrupted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: CI Size: L The size of the change in this PR is large

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Improve and optimize NuttX CI

4 participants