[shaahji]

Add RNNT quantization passes and ONNX MatMulNBits graph surgery

Introduce RNNT-specific optimization paths without requiring transformer-centric model metadata.

• Register RnntSelectiveMixedPrecision and RnntGptq passes.
• Add grouped linear-weight quantization with RTN and shared k-quant parameter selection.
• Resolve group sizes against layer dimensions and persist quantized state dictionaries.
• Keep decoder and joint precision unchanged by ignoring their per-module precision overrides.

Add configurable MatMul-to-MatMulNBits graph conversion:

• Reuse ONNX k-quant packing and canonical initializer naming.
• Support per-node settings, wildcard exclusions, accuracy levels, and progress logging.
• Preserve graph outputs and rewire intermediate consumers.
• Reuse shared quantized initializers and remove unused weights.
• Provide a standalone RemoveUnusedInitializers surgery.

Load Hugging Face configuration before resolving task model classes, retain the required AutoConfig import, and add external-data serialization diagnostics.

Add regression tests for RNNT quantization, shared k-quant parameters, MatMulNBits conversion and naming, initializer reuse and cleanup, and external-data serialization.

Checklist before requesting a review

• Add unit tests for this change.
• Make sure all tests can pass.
• Update documents if necessary.
• Lint and apply fixes to your code by running lintrunner -a
• Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

(Optional) Issue link

[Copilot]

🟡 Changes recommended

Unresolved checkpoint handoff and graph-rewrite defects can discard quantization results or produce invalid ONNX models.

7 open issues

• Prevent incompatible initializer reuse across quantization settings · New
• Replace selected ONNX nodes directly instead of matching names · New
• Rewire renamed outputs captured by nested graph subgraphs · New
• Retain initializers referenced by nested graph subgraphs · New
• Declare the Microsoft opset for MatMulNBits · New
• Return handler for the saved quantized model artifact · New
• Register qparam_strategy in the RnntGptq configuration schema · New

What changed in this PR

This PR adds RNNT-specific quantization paths to Olive without relying on transformer metadata, alongside configurable ONNX MatMulNBits graph conversion.

Changes:

• Register RNNT mixed-precision and grouped weight-quantization passes using RTN or shared k-quant parameters.
• Add MatMulNBits conversion and unused-initializer cleanup, with regression tests.
• Update graph serialization and load Hugging Face configuration before resolving task model classes.

File	Description
test/​passes/​pytorch/​test_rnnt_quantization.py	Tests RNNT metadata and quantized weights.
test/​passes/​onnx/​test_graph_surgeries.py	Tests conversion, initializer reuse, cleanup, and serialization.
olive/​passes/​pytorch/​rnnt_selective_mixed_precision.py	Adds RNNT mixed-precision metadata generation.
olive/​passes/​pytorch/​rnnt_gptq.py	Adds grouped RNNT weight quantization and checkpoint saving.
olive/​passes/​onnx/​graph_surgeries.py	Adds quantization surgeries and switches to IR-based serialization.
olive/​olive_config.json	Registers the RNNT passes.
olive/​common/​hf/​utils.py	Loads model configuration before task-class resolution.

_{Review effort: Balanced}

[github-advanced-security]

lintrunner found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

[Copilot]

🟡 Changes recommended

Critical defects can invalidate ONNX graphs through unsafe value renaming and unsupported block sizes, while quantized checkpoint loading depends on unbundled source resources and can mix CPU weights with CUDA parameters.

5 open issues

• Validate resolved block size for MatMulNBits · New
• Avoid collisions when renaming intermediate outputs · New
• Bundle and rebase resources needed for checkpoint reload · New
• Move quantized weights to the target parameter device · New
• Rewire renamed outputs captured by nested graph subgraphs

1 previously missed issue

In code that hasn't changed since last review

Check shared-weight cache before quantization

olive/​passes/​onnx/​graph_surgeries.py:2280

The shared-weight cache is checked only after _kquant_block_quant has already run. Consequently, every MatMul using the same weight and settings repeats the full k-quant search and packing, then discards the duplicate arrays; only initializer storage is reused. Check (weight_name, node_bits, node_block_size) before quantization and run the expensive work only on a cache miss, especially for large shared weights.

6 resolved since last review

• Return handler for the saved quantized model artifact
• Declare the Microsoft opset for MatMulNBits
• Retain initializers referenced by nested graph subgraphs
• Replace selected ONNX nodes directly instead of matching names
• Prevent incompatible initializer reuse across quantization settings
• Register qparam_strategy in the RnntGptq configuration schema

_{Review effort: Balanced}