Repository navigation
fix(convert): honour <id>:<variant> and --variant in basert pull - #77
Open
MoeEldouma wants to merge 1 commit into
Open
MoeEldouma wants to merge 1 commit into
MoeEldouma wants to merge 1 commit into
Conversation
pull was sending the `:variant` suffix to the Hub as part of the repo name (404) and had no --variant flag, so neither form from the README worked. Strip the variant off the id before resolving and use its quant width for the target, same as the auto-pull path does. Fixes basecompute#70
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #70
basert pull <id>:<variant>was passing the:variantstraight through as part of the HF repo name, hence the 404. Pull also didn't have a--variantflag even though the README mentions one. serve/chat/bench already handle both, so this just makes pull behave the same way.The change is small.
cmd_pullnow strips the variant off the id before resolving (newapply_pull_variantin hub.rs) and uses its quant width to set the target, same asauto_pull_and_resolvedoes. Inline:variantbeats--variantlike it does inresolve_hub_model, and a trailing:or an empty--varianterrors out instead of quietly falling back to q4.--profilestill wins over everything.Added a unit test for those cases.
cargo test -p base-convert, fmt and clippy are all clean on my GB10 (aarch64, Ubuntu 24.04).Before (0.3.0):
After:
One thing I wasn't sure about: the variant only picks the quant width, then the catalog resolver picks the actual build and prefers the backend-native one. So on CUDA
:default-q8ends up installingcuda-q8when there is one (like above). I left it since that's what serve/chat do too, and default-q8 crashes on CUDA anyway (#71). Happy to make it match the exact variant name if you'd rather.