Skip to content

Round smart_resize sizes like Hugging Face - #1140

Merged
Sayan Shaw (sayanshaw24) merged 2 commits into
microsoft:mainfrom
ykhrustalev:ykhrustalev/smart-resize-half-even
Oct 22, 2026
Merged

Sayan Shaw (sayanshaw24) merged 2 commits into
microsoft:mainfrom
ykhrustalev:ykhrustalev/smart-resize-half-even

Conversation

@ykhrustalev

Copy link
Copy Markdown
Contributor

Problem

Resize with smart_resize follows the Hugging Face smart_resize of LFM2-VL and, for sides longer than half the factor, of Qwen2-VL / Qwen2.5-VL / Qwen3-VL, with two differences:

  1. round_by_factor uses std::round, which sends x.5 up. Python's round() sends it to the even integer. With factor 32, a 400 px side becomes 416 instead of 384.
  2. In the min_pixels branch, height * beta and width * beta are cut to integers before the ceil, which can give one factor less than Hugging Face (sizes are width x height: a 30x17 image with the LFM2-VL settings gives 352x192 instead of 352x224).

For example, a 500x400 image with the LFM2-VL settings becomes 512x416, 208 image tokens; transformers' Lfm2VlImageProcessor makes it 512x384, 192 tokens (spatial_shapes [24, 32]). The model gets a different image and a different prompt length than in transformers.

Fix

  • round_by_factor: round half to even, in integer math.
  • ceil_by_factor takes the scaled size as a double, so nothing is cut before the ceil.
  • image_factor_ is set in Init instead of Compute. It only depends on the attributes, and this lets smart_resize be tested on its own; the default (28) is unchanged.
  • Init returns an error when patch_size or merge_size is not positive; the integer rounding would divide by zero.

New test TestSmartResizeMatchesHuggingFace with sizes from transformers, and checks that Init rejects a zero patch_size or merge_size; it fails on main.

Testing

Built with ./build.sh -DOCOS_ENABLE_C_API=ON on Linux (Ubuntu 22.04, gcc 11.4, x86_64) and macOS (27.0.1, Apple clang 21, arm64). The numbers are the same on both machines unless a line says otherwise.

  • ctest: Linux 4/4 targets pass, including the Qwen2.5-VL, Qwen3-VL and Pixtral processor tests. macOS 3/4: pp_api_test fails only in ImageDecoderTest.TestJpegEncoderDecoder, the same as on main.
  • Output size of the ortx processor (C API) on 400 image sizes, 16 to 1024 px per side (30% of the heights are forced to x.5 for factor 32), against transformers 5.20:
Resize settings reference differ, main differ, this PR
LFM2-VL (patch 16, merge 2, 64 to 256 tokens) Lfm2VlImageProcessor.smart_resize 34 (26 larger, 8 smaller) 0
test/data/qwen2.5vl Qwen2-VL smart_resize, factor 28 11 (10 larger, 1 smaller) 0
test/data/qwen3vl Qwen2-VL smart_resize, factor 32 72 (all larger) 0
test/data/pixtral Pixtral resize (longest edge 1540, patch 28); it does not use smart_resize 281 286

The ortx Pixtral config already differs from Hugging Face Pixtral on main. The 10 Pixtral sizes that change all have a side at x.5 of 28, which now rounds down; 5 of them matched Hugging Face before.

  • (Linux) Every size from 1 to 1024 px per side, Resize::smart_resize compiled from the header against the same transformers functions: with both sides longer than half the factor, this PR gives the Hugging Face size for all three settings; main (with image_factor_ set in Init so it can run alone) differs in 24,345 / 36,218 / 30,259 sizes (LFM2-VL / qwen2.5vl / qwen3vl). Sizes with a shorter side still differ: Qwen2-VL rounds that side to 0 before the min_pixels step, and ortx rejects aspect ratios above 200, which Lfm2VlImageProcessor accepts.
  • (Linux) A processor config with patch_size 0: main loads it and OrtxImagePreProcess fails with ImagingResample failed; this PR fails in OrtxCreateProcessor with [Resize]: patch_size and merge_size must be positive (without that check the integer rounding crashes with SIGFPE).

The Hugging Face smart_resize uses Python's round(), which sends x.5 to
the even integer: with factor 32, a 400 px side becomes 384. std::round
sends it up to 416, so a 500x400 image gets 16 more LFM2-VL image tokens
than in transformers. In the min_pixels branch the scaled size was also
truncated to an integer before the ceil, which can give one factor less.
image_factor_ is now set in Init, so smart_resize can be tested on its
own, and Init rejects a patch_size or merge_size that is not positive,
which would divide by zero in the integer rounding.
Copilot AI balanced review requested due to automatic review settings October 19, 2026 17:52
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused arithmetic changes and regression tests address the reported mismatches, with no unresolved review findings.

No open issues 🎉

What changed in this PR

Aligns Resize::smart_resize rounding with Hugging Face image preprocessing.

Changes:

  • Uses half-to-even rounding and preserves fractional sizes before ceiling.
  • Initializes the image factor in Init and rejects nonpositive patch or merge sizes.
  • Adds regression tests for resize dimensions and zero-valued settings.
File Description
test/​pp_api_test/​test_processor.cc Adds rounding and initialization regression tests.
shared/​api/​image_transforms.hpp Corrects rounding and validates factor initialization.

Review effort: Balanced

@kunal-vaishnavi

Copy link
Copy Markdown
Contributor

/azp run onnxruntime-extensions.CI

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@sayanshaw24

Copy link
Copy Markdown
Collaborator

/azp run onnxruntime-extensions.CI

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@sayanshaw24
Sayan Shaw (sayanshaw24) enabled auto-merge (squash) October 22, 2026 22:06
@sayanshaw24
Sayan Shaw (sayanshaw24) merged commit 3251b95 into microsoft:main Oct 22, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants