Skip to content

fix(batch): report short-circuited FIFO records that don't match the model - #8663

Open
soujasK wants to merge 1 commit into
aws-powertools:developfrom
soujasK:fix/fifo-short-circuit-model-error
Open

soujasK wants to merge 1 commit into
aws-powertools:developfrom
soujasK:fix/fifo-short-circuit-model-error

Conversation

@soujasK

@soujasK soujasK commented Oct 25, 2026

Copy link
Copy Markdown

Issue number: closes #8661

Summary

Changes

Once a record fails, SqsFifoPartialProcessor short-circuits the remaining records (or the rest of the failed group with skip_group_on_error=True) and converts them with the customer's model before passing them to failure_handler. That conversion was outside any try, so a record that didn't match the model raised out of process() and failed the whole invocation.

Short-circuited records never reach the record handler, so if the model conversion fails they're now registered as event source data classes, like other poison pills. Records that do match the model are still registered as model instances, so fail_messages doesn't change for them.

Tests: a batch with a successful record, a failing record, a record that doesn't match the model in the same group, and a successful record in another group, parametrized over skip_group_on_error. Both cases raised ValidationError before the change and pass after it. tests/functional/batch passes on Linux (Python 3.12), along with ruff, mypy, ty, bandit and xenon.

User experience

Before: a malformed record after a failure fails the whole invocation, so every record is retried, including already processed ones (and, with skip_group_on_error=True, the successful records of other message groups).

After: the malformed record is reported in batchItemFailures along with the other short-circuited records.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@soujasK
soujasK requested a review from a team as a code owner October 25, 2026 18:05
@soujasK
soujasK requested a review from svozza October 25, 2026 18:05
@boring-cyborg boring-cyborg Bot added the tests label Oct 25, 2026
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Oct 25, 2026
@leandrodamascena leandrodamascena added this to the 3.40.0 milestone Oct 25, 2026
@soujasK
soujasK force-pushed the fix/fifo-short-circuit-model-error branch from 92dc822 to 1545ba0 Compare October 26, 2026 10:41
@sonarqubecloud

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: SqsFifoPartialProcessor fails the whole batch when a short-circuited record doesn't match the model

2 participants