Repository navigation
Round smart_resize sizes like Hugging Face - #1140
Merged
Sayan Shaw (sayanshaw24) merged 2 commits intoOct 22, 2026
Merged
Sayan Shaw (sayanshaw24) merged 2 commits into
Sayan Shaw (sayanshaw24) merged 2 commits into
Conversation
The Hugging Face smart_resize uses Python's round(), which sends x.5 to the even integer: with factor 32, a 400 px side becomes 384. std::round sends it up to 416, so a 500x400 image gets 16 more LFM2-VL image tokens than in transformers. In the min_pixels branch the scaled size was also truncated to an integer before the ceil, which can give one factor less. image_factor_ is now set in Init, so smart_resize can be tested on its own, and Init rejects a patch_size or merge_size that is not positive, which would divide by zero in the integer rounding.
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
3 of 9 tasks
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused arithmetic changes and regression tests address the reported mismatches, with no unresolved review findings.
No open issues 🎉
What changed in this PR
Aligns Resize::smart_resize rounding with Hugging Face image preprocessing.
Changes:
- Uses half-to-even rounding and preserves fractional sizes before ceiling.
- Initializes the image factor in
Initand rejects nonpositive patch or merge sizes. - Adds regression tests for resize dimensions and zero-valued settings.
| File | Description |
|---|---|
| test/pp_api_test/test_processor.cc | Adds rounding and initialization regression tests. |
| shared/api/image_transforms.hpp | Corrects rounding and validates factor initialization. |
Review effort: Balanced
Contributor
|
/azp run onnxruntime-extensions.CI |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
kunal-vaishnavi
approved these changes
Oct 22, 2026
Sayan Shaw (sayanshaw24)
approved these changes
Oct 22, 2026
Collaborator
|
/azp run onnxruntime-extensions.CI |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Sayan Shaw (sayanshaw24)
enabled auto-merge (squash)
October 22, 2026 22:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Resizewithsmart_resizefollows the Hugging Facesmart_resizeof LFM2-VL and, for sides longer than half the factor, of Qwen2-VL / Qwen2.5-VL / Qwen3-VL, with two differences:round_by_factorusesstd::round, which sends x.5 up. Python'sround()sends it to the even integer. With factor 32, a 400 px side becomes 416 instead of 384.min_pixelsbranch,height * betaandwidth * betaare cut to integers before the ceil, which can give one factor less than Hugging Face (sizes are width x height: a 30x17 image with the LFM2-VL settings gives 352x192 instead of 352x224).For example, a 500x400 image with the LFM2-VL settings becomes 512x416, 208 image tokens; transformers'
Lfm2VlImageProcessormakes it 512x384, 192 tokens (spatial_shapes[24, 32]). The model gets a different image and a different prompt length than in transformers.Fix
round_by_factor: round half to even, in integer math.ceil_by_factortakes the scaled size as a double, so nothing is cut before the ceil.image_factor_is set inInitinstead ofCompute. It only depends on the attributes, and this letssmart_resizebe tested on its own; the default (28) is unchanged.Initreturns an error whenpatch_sizeormerge_sizeis not positive; the integer rounding would divide by zero.New test
TestSmartResizeMatchesHuggingFacewith sizes from transformers, and checks thatInitrejects a zeropatch_sizeormerge_size; it fails on main.Testing
Built with
./build.sh -DOCOS_ENABLE_C_API=ONon Linux (Ubuntu 22.04, gcc 11.4, x86_64) and macOS (27.0.1, Apple clang 21, arm64). The numbers are the same on both machines unless a line says otherwise.pp_api_testfails only inImageDecoderTest.TestJpegEncoderDecoder, the same as on main.Lfm2VlImageProcessor.smart_resizetest/data/qwen2.5vlsmart_resize, factor 28test/data/qwen3vlsmart_resize, factor 32test/data/pixtralsmart_resizeThe ortx Pixtral config already differs from Hugging Face Pixtral on main. The 10 Pixtral sizes that change all have a side at x.5 of 28, which now rounds down; 5 of them matched Hugging Face before.
Resize::smart_resizecompiled from the header against the same transformers functions: with both sides longer than half the factor, this PR gives the Hugging Face size for all three settings; main (withimage_factor_set inInitso it can run alone) differs in 24,345 / 36,218 / 30,259 sizes (LFM2-VL / qwen2.5vl / qwen3vl). Sizes with a shorter side still differ: Qwen2-VL rounds that side to 0 before themin_pixelsstep, and ortx rejects aspect ratios above 200, whichLfm2VlImageProcessoraccepts.patch_size0: main loads it andOrtxImagePreProcessfails withImagingResample failed; this PR fails inOrtxCreateProcessorwith[Resize]: patch_size and merge_size must be positive(without that check the integer rounding crashes with SIGFPE).