Skip to content

fix(convert): honour <id>:<variant> and --variant in basert pull - #77

Open
MoeEldouma wants to merge 1 commit into
basecompute:mainfrom
MoeEldouma:fix/pull-variant
Open

MoeEldouma wants to merge 1 commit into
basecompute:mainfrom
MoeEldouma:fix/pull-variant

Conversation

@MoeEldouma

Copy link
Copy Markdown

Fixes #70

basert pull <id>:<variant> was passing the :variant straight through as part of the HF repo name, hence the 404. Pull also didn't have a --variant flag even though the README mentions one. serve/chat/bench already handle both, so this just makes pull behave the same way.

The change is small. cmd_pull now strips the variant off the id before resolving (new apply_pull_variant in hub.rs) and uses its quant width to set the target, same as auto_pull_and_resolve does. Inline :variant beats --variant like it does in resolve_hub_model, and a trailing : or an empty --variant errors out instead of quietly falling back to q4. --profile still wins over everything.

Added a unit test for those cases. cargo test -p base-convert, fmt and clippy are all clean on my GB10 (aarch64, Ubuntu 24.04).

Before (0.3.0):

$ basert pull basecompute/Qwen3-0.6B:default-q8
Error: listing files in basecompute/Qwen3-0.6B:default-q8@main
    1: http status: 404

After:

$ basert pull basecompute/Qwen3-0.6B:default-q8
  catalog: basecompute/Qwen3-0.6B/Qwen3-0.6B-cuda-q8.base@main (pre-converted)
  sha256:  verified
installed basecompute/Qwen3-0.6B [cuda-q8] → …/basecompute/Qwen3-0.6B/cuda-q8/model.base

$ basert pull basecompute/Qwen3-8B --variant default-q8 --dry-run
plan: pull pre-converted basecompute/Qwen3-8B [default-q8] from basecompute/Qwen3-8B/Qwen3-8B-Q8.base@main (no conversion)

One thing I wasn't sure about: the variant only picks the quant width, then the catalog resolver picks the actual build and prefers the backend-native one. So on CUDA :default-q8 ends up installing cuda-q8 when there is one (like above). I left it since that's what serve/chat do too, and default-q8 crashes on CUDA anyway (#71). Happy to make it match the exact variant name if you'd rather.

pull was sending the `:variant` suffix to the Hub as part of the repo
name (404) and had no --variant flag, so neither form from the README
worked. Strip the variant off the id before resolving and use its quant
width for the target, same as the auto-pull path does.

Fixes basecompute#70

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

basert pull <id>:<variant> returns 404 (README says it selects the quant build)

1 participant