feat(server): add opt-in bounded batch progress

[svonava]

Continuous cheap arrivals can repeatedly bypass an older expensive item in the Python worker's cost-sorted batcher. This adds an explicit bounded_batch_progress profile option, disabled by default. When enabled, a bypassed oldest item is reserved for the next batchable turn. An item with k older batchable items receives a dispatch opportunity within 2 * (k + 1) nonempty batchable turns of its batcher; isolated-item turns may interleave.

The ordinary BatchFormer source and pending-request layout remain unchanged. The worker selects the enabled class when constructed, including for lazy LoRA batchers. Omitted or null values inherit the parent setting; explicit false disables an enabled parent. The option is validated and carried through the config-service schema, YAML resolution and model loader. Existing profiles keep the same serialized settings; enabling the option contributes to profile identity.

Enabled scheduling changes batch composition and can change model scores, labels and throughput. The earlier #798 GLiClass qualification failed its original 0.002 numerical bound: maximum score difference 0.0232588 with 26 changed choices. This draft retains that failure, enables no shipped profile and does not establish numerical equivalence or close #390. The dispatch bound is not a wall-clock deadline; preformed and sidecar batches bypass this policy.

Validation:

• 108 focused cases and an affected regression suite with 422 passing tests and one existing missing-flash-attn skip; Ruff, formatting and scoped type checks pass.
• All 214 existing model config dumps and 377 resolved profile dumps match the baseline. The original batcher bytes are unchanged, and enabled extraction matches fix(batching): guarantee progress for bypassed requests #798 across 2,000 randomized dispatch turns.
• Independent review passed 36 additional controls covering configuration, typed schemas, default-path selection, progress bounds, mixed isolated/batchable work, conservation and cleanup.

These are CPU and source-level checks; this wrapper has no new GPU performance or model-quality qualification. The PR remains draft while review and the compatibility decision are pending. Related to #390; #798 remains unchanged.

Summary by CodeRabbit

• New Features
  • Added an opt-in profile setting that helps older batchable requests get dispatched within a bounded number of batchable turns, even when newer, lower-cost requests arrive.
  • The setting defaults to off and is not enabled in shipped profiles. Existing request and cost limits remain in effect.
• Documentation
  • Documented how the setting is inherited and how enabling it may affect batch composition, throughput, scores, or selected labels.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Team
• Run ID: 90d101b4-df6d-4ae6-8ace-d2c49bc460d1

📥 Commits

Reviewing files that changed from the base of the PR and between 118ba91 and 30d38e5.

📒 Files selected for processing: 10

• packages/sie_config/src/sie_config/model_config.schema.json
• packages/sie_server/README.md
• packages/sie_server/src/sie_server/config/model.py
• packages/sie_server/src/sie_server/core/loader.py
• packages/sie_server/src/sie_server/core/model_loader.py
• packages/sie_server/src/sie_server/core/progress_batcher.py
• packages/sie_server/src/sie_server/core/worker/model_worker.py
• packages/sie_server/src/sie_server/core/worker/types.py
• packages/sie_server/tests/config/test_batch_progress.py
• packages/sie_server/tests/core/test_progress_batcher.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.

📝 Walkthrough

Adds an opt-in bounded_batch_progress profile setting and a progress-aware batcher for Python workers. The setting inherits through profile extensions and defaults to disabled. When enabled, the batcher reserves bypassed requests for later batchable turns while retaining existing request, cost, and item limits.

Changes: Bounded batch progress

Layer / File(s)	Summary
Profile setting and resolution packages/sie_config/src/sie_config/model_config.schema.json, packages/sie_server/src/sie_server/config/model.py, packages/sie_server/src/sie_server/core/loader.py, packages/sie_server/tests/config/test_batch_progress.py	Adds the nullable setting, parent inheritance, serialization behavior, and schema validation. Generated variants preserve enabled values. Tests cover validation, inheritance, serialization, and configuration loading.
Progress-aware batch formation packages/sie_server/src/sie_server/core/progress_batcher.py, packages/sie_server/tests/core/test_progress_batcher.py, packages/sie_server/README.md	Adds ProgressBatchFormer, which tracks request arrival order and reserves a bypassed batchable request for a later turn. Tests cover progress bounds, reservation behavior, limits, cleanup, and compatibility. The README documents the setting and scheduling behavior.
Worker batcher selection packages/sie_server/src/sie_server/core/worker/types.py, packages/sie_server/src/sie_server/core/model_loader.py, packages/sie_server/src/sie_server/core/worker/model_worker.py, packages/sie_server/tests/config/test_batch_progress.py, packages/sie_server/tests/core/test_progress_batcher.py	Passes the resolved setting into WorkerConfig. The worker selects ProgressBatchFormer for base-model and LoRA batchers when enabled, and retains BatchFormer when disabled.

Sequence Diagram(s)

sequenceDiagram
  participant ProfileResolution
  participant ModelLoader
  participant WorkerConfig
  participant ModelWorker
  participant ProgressBatchFormer
  ProfileResolution->>ModelLoader: Provide resolved bounded_batch_progress
  ModelLoader->>WorkerConfig: Set bounded_batch_progress
  WorkerConfig->>ModelWorker: Configure worker
  ModelWorker->>ProgressBatchFormer: Construct batcher when enabled

Loading

Suggested reviewers: krisztian-gajdar

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 30d38

The change adds an opt-in batching option that is off by default and leaves existing behavior unchanged unless a profile enables it. No concrete merge-blocking risk was found. The PR remains a draft, and the author has already said the compatibility decision is pending.

Pre-merge checks: 3 · 2

Failed checks: 2 warnings

Check name	Status	Explanation	Resolution
Linked Issues check		Direct issue [#390] asks for investigation of separate-label-group request splitting and scheduling, timeout or cancellation visibility, and reproduction under mixed extract load. This PR adds an opt-…	Add automated coverage for the reported separate-label-group and mixed-extract workload. Add coverage or implementation for the required per-request timeout and cancellation visibility. If this PR intentionally addresses only scheduler prog…
Docstring Coverage		Docstring coverage is 21.62% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 37 functions across 8 files. (2 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 3 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: adding an opt-in bounded batch progress feature for the server.
Out of Scope Changes check		The new ProgressBatchFormer, worker selection, profile inheritance, schema and serialization changes, loader plumbing, documentation, and tests support the opt-in scheduling objective described for [#…

Full details: Out of Scope Changes check

Explanation

The new ProgressBatchFormer, worker selection, profile inheritance, schema and serialization changes, loader plumbing, documentation, and tests support the opt-in scheduling objective described for [#390]. The changes preserve the default batcher and do not enable the option in shipped profiles. No unrelated change is established by the reviewed diff.

Full details: Linked Issues check

Explanation

Direct issue [#390] asks for investigation of separate-label-group request splitting and scheduling, timeout or cancellation visibility, and reproduction under mixed extract load. This PR adds an opt-in bounded-progress scheduler and unit tests for cost-sorted batch progress. The reviewed changes do not add a GLiClass separate-group split or mixed-load reproduction, and they do not add or test per-request timeout or cancellation visibility. The PR description also states that it does not close [#390].

Resolution

Add automated coverage for the reported separate-label-group and mixed-extract workload. Add coverage or implementation for the required per-request timeout and cancellation visibility. If this PR intentionally addresses only scheduler progress, remove [#390] as a direct linked target or split the remaining investigation into a separate change.

Full details: Docstring Coverage

Explanation

Docstring coverage is 21.62% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 37 functions across 8 files. (2 skipped: 2 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

• Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[svonava]

@coderabbitai full review

Please review exact head 30d38e5f20b5794b55f38470618b1416f048fb69, including disabled-path preservation, profile inheritance and serialization, config-service validation, and opt-in progress/cleanup. This remains a draft with the prior numerical failure and compatibility decision explicitly retained.

[coderabbitai]

✅ Action performed

Full review finished.