From 5a71cd36037ba3561326c05346f5f3714614e04e Mon Sep 17 00:00:00 2001 From: Austin Dimmer Date: Mon, 26 Oct 2026 12:55:15 +0000 Subject: [PATCH 1/2] fix(INFIAAS-11916): jenkins-agent-terraform-2408 Node 24 LTS, eMudhra/emSign CA refresh, Go 1.26.10 The eMudhra/emSign incident (eight compromised intermediates) needs every CA store in the agent refreshed: pip always loads its vendored certifi in addition to the system store, so OS ca-certificates alone is not enough. - NODEJS_VERSION 20 -> 24 (Node 20 is EOL; 24.22.1 ships the fixed root store) - uv 0.10.9 -> 0.13.5 (embedded CA store fix) - pip >= 26.2.2 and certifi >= 2026.10.23 for every uv-managed Python - dnf update ca-certificates in the package layer - GO_VERSION 1.21.13 -> 1.26.10, and the tarball's SHA-256 is verified (ARG GO_SHA256, from go.dev) - drop Python 3.9: EOL, and no pip release with the fix supports it (pip >= 26.1 requires Python >= 3.10). uv is installed, so users can still `uv python install 3.9` - the default Python (PYTHON_DEFAULT_VERSION=3.12, defined once) becomes python3, python and python3-config (an exec wrapper, because the script derives its prefix from its own path), and its console scripts (pipenv, virtualenv, az, pre-commit, ...) are linked into /usr/local/bin, which now precedes /usr/bin on PATH. Before, python3 and these scripts had the agent's libraries only because `--system --python 3.9` resolved to the RHEL platform python3.9. The build asserts python3, its libraries, python3-config, the CLIs, and that dnf still works on the platform python3 - the Python loop now fails the build on the first interpreter that fails; before, `fi;` let a later successful iteration mask an earlier failure - CHANGELOG: the python3/pipenv default under Changed, the Python 3.9 removal under Removed - docs: Node/Python/Go versions, python3, tenv (not tfenv) for 2408 --- CHANGELOG.md | 5 ++ .../terraform-2408/docker/Dockerfile.ubi9 | 62 ++++++++++++++++--- .../terraform-2408/docker/python_requirements | 1 + .../jenkins-agents/pages/inf-terraform.adoc | 5 +- 4 files changed, 62 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 441fd655..f6860c3d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,8 +4,13 @@ ### Added ### Changed +- jenkins-agent-terraform-2408: `python3`, `python` and `python3-config` are now the uv-managed default Python 3.12 with the agent's libraries (before: the RHEL platform Python 3.9), and pipenv creates Python 3.12 environments by default (before: 3.9) ([#NNNN](https://github.com/opendevstack/ods-quickstarters/pull/NNNN)) ### Fixed +- jenkins-agent-terraform-2408: Node 24 LTS, CA store refresh for eMudhra/emSign (pip>=26.2.2, certifi>=2026.10.23, ca-certificates, uv 0.13.5), Go 1.26.10 with SHA-256 check ([#NNNN](https://github.com/opendevstack/ods-quickstarters/pull/NNNN)) + +### Removed +- jenkins-agent-terraform-2408: Python 3.9 (EOL, and no pip release with the CA fix supports it); install it on demand with `uv python install 3.9` ([#NNNN](https://github.com/opendevstack/ods-quickstarters/pull/NNNN)) ## [4.13.1] - 2026-07-30 ### Fixed diff --git a/common/jenkins-agents/terraform-2408/docker/Dockerfile.ubi9 b/common/jenkins-agents/terraform-2408/docker/Dockerfile.ubi9 index b8f5118a..017a5a9c 100644 --- a/common/jenkins-agents/terraform-2408/docker/Dockerfile.ubi9 +++ b/common/jenkins-agents/terraform-2408/docker/Dockerfile.ubi9 @@ -16,6 +16,9 @@ ARG nexusHost ARG nexusUrl=https://${nexusHost} ARG nexusAuth ARG NPMJS_REGISTRY_FLAG +# SHA-256 of go${GO_VERSION}.linux-amd64.tar.gz, from https://go.dev/dl/?mode=json&include=all. +# Change it together with GO_VERSION. +ARG GO_SHA256=ccfe7ae822d2c2f41f4fe7c956199967a4dac22d0ee0d2b2bfdb440aab3b20c5 ENV TERRAFORM_VERSION=1.9.4 \ TERRAFORM_CONFIG_INSPECT_VERSION=0.2.0 \ @@ -26,17 +29,18 @@ ENV TERRAFORM_VERSION=1.9.4 \ TENV_VERSION=3.0.0 \ TENV_AUTO_INSTALL=true \ TENV_ROOT=/opt/tenv \ - GO_VERSION=1.21.13 \ + GO_VERSION=1.26.10 \ GOBIN=/usr/local/go/bin \ TFLINT_VERSION=0.52.0 \ - NODEJS_VERSION=20 \ + NODEJS_VERSION=24 \ BUNDLER_VERSION=2.5.17 \ AGE_VERSION=1.2.0 \ GEM_HOME=/opt/bundle \ RBENV_ROOT=/opt/rbenv \ RBENV_SHELL=bash \ TASK_VERSION=3.38.0 \ - UV_VERSION=0.10.9 \ + UV_VERSION=0.13.5 \ + PYTHON_DEFAULT_VERSION=3.12 \ NPM_CONFIG_CAFILE=/etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem \ UV_CA_CERTIFICATE=/etc/ssl/certs/ca-bundle.crt \ PIP_CERT=/etc/ssl/certs/ca-bundle.crt @@ -46,7 +50,9 @@ ENV INSTALL_PKGS="yum-utils gcc make git-core zlib zlib-devel gcc-c++ patch \ libffi-devel libyaml-devel openssl-devel make bzip2 autoconf curl sqlite-devel xz \ jq parallel" ENV INSTALL_DNF_PKGS="readline-devel bison automake libtool xorriso" -ENV PATH=/opt/rbenv/shims:/opt/rbenv/bin:/opt/node/bin:/usr/bin:/usr/local/go/bin:$HOME/.local/bin:$JAVA_HOME/bin:$PATH +# /usr/local/bin comes before /usr/bin so that python3/python resolve to the default Python +# (linked below), not to the RHEL platform python3 (dnf keeps using it via its shebang). +ENV PATH=/opt/rbenv/shims:/opt/rbenv/bin:/opt/node/bin:/usr/local/bin:/usr/bin:/usr/local/go/bin:$HOME/.local/bin:$JAVA_HOME/bin:$PATH ENV HOME=/home/jenkins RUN sh -c "rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm || true" @@ -65,7 +71,9 @@ RUN sed -i 's@^\s*enabled\s*=.*$@enabled = 1@g' /etc/yum.repos.d/*.repo \ RUN set -x \ && dnf -y repolist \ && dnf -y install --allowerasing $INSTALL_PKGS \ - && dnf -y install --allowerasing --enablerepo almalinux-baseos --enablerepo almalinux-appstream $INSTALL_DNF_PKGS + && dnf -y install --allowerasing --enablerepo almalinux-baseos --enablerepo almalinux-appstream $INSTALL_DNF_PKGS \ + && dnf -y update ca-certificates \ + && dnf clean all RUN curl -LsSf https://astral.sh/uv/${UV_VERSION}/install.sh | sh && \ mkdir -p $HOME/.config/pip && \ @@ -75,23 +83,58 @@ RUN curl -LsSf https://astral.sh/uv/${UV_VERSION}/install.sh | sh && \ echo "trusted-host = ${nexusHost}" >> $HOME/.config/pip/pip.conf; \ fi -RUN for pyVersion in 3.9 3.11 3.12; do \ +# Python 3.9 is not installed any more: it is EOL and pip >= 26.2.2 (the release with the +# refreshed vendored CA store, see eMudhra/emSign incident) requires Python >= 3.10. +# Users who still need it can run `uv python install 3.9` themselves. +# pip is upgraded explicitly because pip always loads its vendored certifi CA store in +# addition to the system store; the floor makes the build fail if it cannot be met. +# The default Python (PYTHON_DEFAULT_VERSION) becomes `python3`, `python` and `python3-config`, and +# its console scripts (pipenv, virtualenv, az, pre-commit, ...) are linked into /usr/local/bin. +# Before, `python3` and these scripts had the agent's libraries only as a side effect of +# `--system --python 3.9` resolving to the RHEL platform /usr/bin/python3.9; a uv-managed +# interpreter keeps its scripts in its own bin dir, which is not on PATH. /usr/bin/python3 stays +# the platform Python, which dnf uses through its shebang; the build checks that dnf still works. +# python3-config is an exec wrapper, not a symlink: the script derives its prefix from its own path. +RUN for pyVersion in 3.11 ${PYTHON_DEFAULT_VERSION}; do \ uv python install $pyVersion && \ if [ -n "${nexusHost}" ] && [ -n "${nexusAuth}" ]; then \ uv pip install --system --break-system-packages --python $pyVersion --index-url https://${nexusAuth}@${nexusHost}/repository/pypi-all/simple --trusted-host ${nexusHost} --prerelease=allow \ + "pip>=26.2.2" \ virtualenv==20.26.3 \ pycodestyle==2.12.1 \ -r /tmp/requirements.txt; \ else \ uv pip install --system --break-system-packages --python $pyVersion --prerelease=allow \ + "pip>=26.2.2" \ virtualenv==20.26.3 \ pycodestyle==2.12.1 \ -r /tmp/requirements.txt; \ - fi; \ + fi && \ + "$(uv python find $pyVersion)" -m pip --version && \ + "$(uv python find $pyVersion)" -c "import certifi; print('certifi', certifi.__version__)" || exit 1; \ done && \ rm -f /tmp/requirements.txt && \ - uv python pin 3.12 && \ - uv tool update-shell + uv python pin ${PYTHON_DEFAULT_VERSION} && \ + uv tool update-shell && \ + PY_DEFAULT="$(uv python find ${PYTHON_DEFAULT_VERSION})" && \ + PY_SCRIPTS="$("$PY_DEFAULT" -c 'import sysconfig; print(sysconfig.get_path("scripts"))')" && \ + ln -sf "$PY_DEFAULT" /usr/local/bin/python3 && \ + ln -sf "$PY_DEFAULT" /usr/local/bin/python && \ + printf '#!/bin/sh\nexec "%s" "$@"\n' "$PY_SCRIPTS/python${PYTHON_DEFAULT_VERSION}-config" >/usr/local/bin/python3-config && \ + chmod 755 /usr/local/bin/python3-config && \ + for script in "$PY_SCRIPTS"/*; do \ + name="$(basename "$script")"; \ + case "$name" in python*|pip|pip3|pip3.*|idle*|pydoc*|2to3*) continue ;; esac; \ + [ -e "/usr/local/bin/$name" ] || ln -s "$script" "/usr/local/bin/$name" || exit 1; \ + done && \ + [ "$(command -v python3)" = /usr/local/bin/python3 ] && \ + python3 --version | grep -q "^Python ${PYTHON_DEFAULT_VERSION}\." && \ + python3 -c 'import boto3, botocore, requests, hcl2, certifi' && \ + PY_INCLUDE="$(python3 -c 'import sysconfig; print(sysconfig.get_paths()["include"])')" && \ + [ -f "$PY_INCLUDE/Python.h" ] && \ + [ "$(python3-config --includes | awk '{print $1}')" = "-I$PY_INCLUDE" ] && \ + pipenv --version && virtualenv --version && pre-commit --version && pycodestyle --version && command -v az && \ + /usr/bin/python3 -c 'import dnf' && dnf --version # Install awscli2 RUN curl -sSL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \ @@ -137,6 +180,7 @@ RUN mkdir -p "${TENV_ROOT}" && chmod 2775 "${TENV_ROOT}" && chown 1001:0 "${TENV # Install GO for terratest RUN cd /tmp \ && curl -LfSso /tmp/go.tar.gz https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz \ + && echo "${GO_SHA256} /tmp/go.tar.gz" | sha256sum -c - \ && tar -C /usr/local -xzf go.tar.gz \ && rm -f /tmp/go.tar.gz \ && /usr/local/go/bin/go version diff --git a/common/jenkins-agents/terraform-2408/docker/python_requirements b/common/jenkins-agents/terraform-2408/docker/python_requirements index e546ab4c..be63367d 100644 --- a/common/jenkins-agents/terraform-2408/docker/python_requirements +++ b/common/jenkins-agents/terraform-2408/docker/python_requirements @@ -8,3 +8,4 @@ pipenv~=2024.0 python-hcl2~=2.0 pre-commit~=3.8.0 azure-cli~=2.63 +certifi>=2026.10.23 diff --git a/docs/modules/jenkins-agents/pages/inf-terraform.adoc b/docs/modules/jenkins-agents/pages/inf-terraform.adoc index baae869d..758c4c5a 100644 --- a/docs/modules/jenkins-agents/pages/inf-terraform.adoc +++ b/docs/modules/jenkins-agents/pages/inf-terraform.adoc @@ -5,14 +5,15 @@ This agents are used to deploy (Cloud) resources using terraform code. There are two terraform jenkins agents available: 1. jenkins-agent-terraform-2306 (ruby 3.2.2, terraform 1.4.6, and matching tools). -1. jenkins-agent-terraform-2408 (ruby 3.3.4, terraform 1.9.4, and matching tools) - this is the current active jenkins agent. +1. jenkins-agent-terraform-2408 (ruby 3.3.4, terraform 1.9.4, Node.js 24 LTS, Python 3.11 and 3.12 (default, also `python3`), Go 1.26, and matching tools) - this is the current active jenkins agent. The images are built in the global `ods` project and are named `jenkins-agent-terraform-2306` and `jenkins-agent-terraform-2408`. It can be referenced in a `Jenkinsfile` with e.g. `ods/jenkins-agent-terraform-2306:latest` or `ods/jenkins-agent-terraform-2408:latest`. == Features -1. support of multiple terraform versions using tfenv +1. support of multiple terraform versions using tfenv (jenkins-agent-terraform-2306) or tenv (jenkins-agent-terraform-2408) 2. integrated test-framework using kitchen-terraform and cinc-auditor +3. Python versions managed by uv in jenkins-agent-terraform-2408 (default 3.12, which is also `python3` and `python` and has the agent's Python libraries; pipenv creates 3.12 environments by default); other versions, e.g. the EOL 3.9, can be installed with `uv python install `. `/usr/bin/python3` is the RHEL platform Python, kept for the OS tools only. == Known limitations n/a From a1824be6a88c031f0daef2334a2b4fc4210b393e Mon Sep 17 00:00:00 2001 From: Austin Dimmer Date: Mon, 26 Oct 2026 15:05:10 +0000 Subject: [PATCH 2/2] docs(INFIAAS-11916): link CHANGELOG entry to #1177 --- CHANGELOG.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f6860c3d..d3c05d88 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,13 +4,13 @@ ### Added ### Changed -- jenkins-agent-terraform-2408: `python3`, `python` and `python3-config` are now the uv-managed default Python 3.12 with the agent's libraries (before: the RHEL platform Python 3.9), and pipenv creates Python 3.12 environments by default (before: 3.9) ([#NNNN](https://github.com/opendevstack/ods-quickstarters/pull/NNNN)) +- jenkins-agent-terraform-2408: `python3`, `python` and `python3-config` are now the uv-managed default Python 3.12 with the agent's libraries (before: the RHEL platform Python 3.9), and pipenv creates Python 3.12 environments by default (before: 3.9) ([#1177](https://github.com/opendevstack/ods-quickstarters/pull/1177)) ### Fixed -- jenkins-agent-terraform-2408: Node 24 LTS, CA store refresh for eMudhra/emSign (pip>=26.2.2, certifi>=2026.10.23, ca-certificates, uv 0.13.5), Go 1.26.10 with SHA-256 check ([#NNNN](https://github.com/opendevstack/ods-quickstarters/pull/NNNN)) +- jenkins-agent-terraform-2408: Node 24 LTS, CA store refresh for eMudhra/emSign (pip>=26.2.2, certifi>=2026.10.23, ca-certificates, uv 0.13.5), Go 1.26.10 with SHA-256 check ([#1177](https://github.com/opendevstack/ods-quickstarters/pull/1177)) ### Removed -- jenkins-agent-terraform-2408: Python 3.9 (EOL, and no pip release with the CA fix supports it); install it on demand with `uv python install 3.9` ([#NNNN](https://github.com/opendevstack/ods-quickstarters/pull/NNNN)) +- jenkins-agent-terraform-2408: Python 3.9 (EOL, and no pip release with the CA fix supports it); install it on demand with `uv python install 3.9` ([#1177](https://github.com/opendevstack/ods-quickstarters/pull/1177)) ## [4.13.1] - 2026-07-30 ### Fixed