Repository navigation
Run Python matrix jobs in parallel - #667
Open
Aasav Badera (aasavbadera) wants to merge 4 commits into
Open
Aasav Badera (aasavbadera) wants to merge 4 commits into
Aasav Badera (aasavbadera) wants to merge 4 commits into
Conversation
Collaborator
Author
|
E2E validation is running on commit Azure pipeline 20261116.6 / build 191449502 This run uses the existing pipeline and trusted Azure service connection to test all five Python versions with |
Allow all five Python versions to run concurrently and update the contributor documentation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 228686cb-da70-40cd-afb9-bb07f8af664f
Keep eight-worker unit testing and five parallel Python matrix jobs, run E2E tests separately with one worker, and publish both JUnit reports together. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 228686cb-da70-40cd-afb9-bb07f8af664f
Retry replayable transient Kusto requests in E2E tests, preserve unsafe-write and permanent-error failures, and bound each CI E2E process group to 25 minutes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 228686cb-da70-40cd-afb9-bb07f8af664f
Aasav Badera (aasavbadera)
force-pushed
the
aasavbadera/parallelism
branch
from
November 16, 2026 11:18
bc8f161 to
8b9b4ce
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 228686cb-da70-40cd-afb9-bb07f8af664f
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block the Python CI parallelization and test-only retry changes.
No open issues 🎉
What changed in this PR
This PR reduces Python CI runtime by running the five-version matrix concurrently, with test-only request retries and an E2E timeout to manage transient failures without changing production SDK behavior.
Changes:
- Run up to five matrix jobs concurrently, using eight workers for each suite and separate JUnit reports merged for publishing.
- Add replay-aware request retries with exponential backoff, jitter, and
Retry-Aftersupport, plus a 25-minute CI E2E deadline. - Propagate row-count request failures and add regression coverage for retry behavior.
| File | Description |
|---|---|
pyproject.toml |
Adds an eight-worker E2E task and separate report paths. |
e2e_retry.py |
Implements synchronous and asynchronous replay-aware retries. |
CONTRIBUTING.md |
Documents concurrency, retries, timeouts, and reporting. |
conftest.py |
Applies request retries only to E2E modules, including setup. |
azure-kusto-ingest/tests/test_e2e_retry.py |
Tests retry limits, safety, delays, and stream replay. |
azure-kusto-ingest/tests/test_e2e_ingest.py |
Propagates row-count failures and tests retry exhaustion. |
.gitignore |
Ignores generated test reports. |
.azuredevops/build.yml |
Enables parallel jobs, bounds E2E runtime, and merges reports. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Retry-After.Validation
Latest validated head:
8b9b4ce92d313d7c2aa7c21db529eeb341a69b49.Azure pipeline 20261116.12 / build 191465162 succeeded against this exact commit.
createdOnto the latestlastChangedOn, not retry-sensitive top-level build timestamps.The branch was rebased onto updated
mastercontaining #666 while the preceding run was executing. All seven files in this retry/concurrency/timeout update were unchanged by that rebase. The current baseline includes #666's local HTTP redirect fixtures, so unit/E2E counts differ from earlier runs. The preceding build 191464067 also passed onbc8f1619fe6813e25caacdf1ded8219ec149f5bb, with 6.84 minutes of log activity; the newer run above validates the actual rebased PR head.Focused local validation of the retry implementation, before that rebase, also passed:
git diff --checkpassed.Prior observed timings
The original five-job/eight-worker run, build 191449502, passed in 5.32 minutes of log activity. The subsequent one-worker E2E run, build 191454946, passed in 11.82 minutes. This update restores the higher worker count with targeted backoff and an overall deadline instead of the one-worker restriction.
For context, the successful serial baseline build 191431977 took 34.21 minutes. These are observed log-index spans, not controlled benchmarks; the serial master run also generated documentation.
Scope and limitations
Retry limits apply to individual wrapped Kusto transport invocations. Existing higher-level SDK retry/fallback policies remain unchanged; the 25-minute deadline bounds the complete CI E2E invocation. The fixture is scoped to the two E2E modules, including their class setup, and does not change ordinary unit-test or production SDK behavior.
Concurrency is per pipeline run, not a global budget or strict request-rate limit. Concurrent builds still share backend capacity. Hard termination may interrupt fixture cleanup and leave temporary test tables.