Repository navigation
feat(kingfisher): Use Python 3.13 for Python apps on kingfisher-main - #683
Merged
Merged
Conversation
Drop pythonX.Y-distutils, which the deadsnakes PPA doesn't build for 3.12+. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QSfuGf81zPF23KcUdGhEaF
…hanges virtualenv.managed has no mod_watch function, so the watch requisite on pkg: python didn't recreate it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QSfuGf81zPF23KcUdGhEaF
A watch requisite on virtualenv.managed doesn't recreate the virtual environment, because the state has no mod_watch function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QSfuGf81zPF23KcUdGhEaF
This was referenced Oct 30, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Python 3.10 reaches end-of-life on 2026-10-31. kingfisher-main's Python apps (Scrapyd, kingfisher-collect's incremental crawls, kingfisher-summarize and collect-generic) build their virtualenvs from the server's default Python, which is 3.10 on Ubuntu 22.04. This PR sets the
python:versionpillar, so their virtualenvs use Python 3.13 from the deadsnakes PPA, independently of the OS. 3.13 is Ubuntu 26.04's default (#602).pillar/kingfisher_main.sls: Setpython:versionto'3.13'.salt/python/init.sls: Droppython{version}-distutils, which deadsnakes doesn't build for 3.12+ (distutils was removed from the standard library). No other pillar setspython:version, so nothing else uses it.salt/lib.sls: Recreate the virtual environment if the Python version changes. Thevirtualenvmacro'swatch: pkg: pythondidn't do this:virtualenv.managedhas nomod_watchfunction, so the watch only ordered the states, and an existing.vekept its old Python. Afile.absenton.ve, withonchanges: pkg: python, now removes it only whenpkg: pythonreports changes (i.e. when a newpythonX.Ypackage is installed), before thevirtualenv.managedstates recreate it. The existingonchangeson-requirementsthen reinstalls the requirements.Testing
In an
ubuntu:22.04container, with the deadsnakes PPA:python3.13andpython3.13-dev(forpython.psycopg) are available.python3.13-distutilsisn't.virtualenv --python=/usr/bin/python3.13, as thevirtualenvmacro runs it, succeeds andpip install uvworks. Ubuntu'svirtualenv20.13 has no bundled pip for 3.13 (python3-pip-whlis pip 22.0.2), so it downloads the latest pip, setuptools and wheel from PyPI. kingfisher-main already installs requirements from PyPI. Offline, the virtualenv creation fails../run.py kingfisher-main state.apply test=Truerenders the four*-virtualenv-clearstates. In test mode,pkg.installedforpython3.13raises an exception (the PPA isn't actually added in test mode), so its dependent states report "requisite failed".Deploy
Deploy while no Scrapyd crawls are running: recreating Scrapyd's virtualenv restarts Scrapyd. A second
test=Truerun should then report the*-virtualenv-clearstates as not run.Follow-up
.python-version(and related settings) in collect-generic, kingfisher-collect and kingfisher-summarize to 3.13.Part of open-contracting/software-development-handbook#153.
🤖 Generated with Claude Code
https://claude.ai/code/session_01QSfuGf81zPF23KcUdGhEaF