common: fix explicit -md precedence over draft sidecar resolution

[ServeurpersoCom]

Overview

An explicit --model-draft file given with -hfd was silently overridden by the sidecar resolution of the draft repo, and its path was never resolved to a local file.

1. An explicit draft file selection now disables the sidecar resolution, so the manual CLI configuration wins over the automatic one.

2. The -hfd tag now also applies to the sidecar resolution directly, so requesting a quant that only exists as a sidecar works instead of failing on the missing full model.

Additional information

Follow-up of #25955

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES

[ggerganov]

For my understanding, when the logic resolves the sidecar automatically how does it determine which quantization to pick up? For example this command:

llama serve \
  --hf-repo       ggml-org/Qwen3.6-27B-GGUF:Q8_0 \
  --hf-repo-draft ggml-org/Qwen3.6-27B-GGUF \
  --spec-type draft-mtp

will pick the mtp-Qwen3.6-27B-Q4_0.gguf, while there is also a mtp-Qwen3.6-27B-Q8_0.gguf in the repo: https://huggingface.co/ggml-org/Qwen3.6-27B-GGUF/tree/main.

Also, this command works:

# request Q8_0 MTP
llama serve \
  --hf-repo       ggml-org/Qwen3.6-27B-GGUF:Q8_0 \
  --hf-repo-draft ggml-org/Qwen3.6-27B-GGUF:Q8_0 \
  --spec-type draft-mtp

but this command fails:

# request Q4_0 MTP
llama serve \
  --hf-repo       ggml-org/Qwen3.6-27B-GGUF:Q8_0 \
  --hf-repo-draft ggml-org/Qwen3.6-27B-GGUF:Q4_0 \
  --spec-type draft-mtp

# error:
# 0.01.122.954 E llama_model_load_from_file_impl: exactly one out metadata, path_model, and file must be defined

[ServeurpersoCom]

The sidecar quant is anchored on the draft plan primary, so no tag falls back to the default Q4_K_M hence mtp-Q4_0, and :Q4_0 fails because it first requires a full Q4_0 model to exist, the tag should apply to the sidecar directly when a spec type is requested, fixing it

[ggerganov]

Let's promote these logs from trace to info for consistency with the mmproj and primary:

diff --git a/common/speculative.cpp b/common/speculative.cpp
index 3cb08767b..ee94d7c37 100644
--- a/common/speculative.cpp
+++ b/common/speculative.cpp
@@ -2284,7 +2284,7 @@ common_speculative_init_result::common_speculative_init_result(
     std::string model_path;
     if (has_draft) {
         model_path = params.speculative.draft.mparams.path;
-        LOG_TRC("%s: loading draft model '%s'\n", __func__, model_path.c_str());
+        LOG_INF("%s: loading draft model '%s'\n", __func__, model_path.c_str());
 
         llama_model * model_dft = llama_model_load_from_file(params.model.path.c_str(), mparams);
         if (model_dft == NULL) {
@@ -2304,7 +2304,7 @@ common_speculative_init_result::common_speculative_init_result(
     } else if (spec_mtp) {
         model_path = params.model.path;
 
-        LOG_TRC("%s: creating MTP draft context against the target model '%s'\n", __func__, model_path.c_str());
+        LOG_INF("%s: creating MTP draft context against the target model '%s'\n", __func__, model_path.c_str());
 
         llama_context * ctx_dft = llama_init_from_model(model_tgt, cparams);
         if (ctx_dft == nullptr) {

Seeing what data is loaded is useful even in -lv 3.

[ggerganov]

We should think about adding unit tests for a few common configurations to check that the correct files are being downloaded. We'll soon add dspark support and it might cause some extra stir in the sidecar resolution implementation.

[ServeurpersoCom]

What do you think about testing on synthetic repo layouts, hardcoded file listings mimicking real repos like Qwen etc..., so the resolution and the task assembly are asserted without the flakiness of CI internet access, this also lets us cover a large amount of configurations, close to an exhaustive check of the resolution logic ?

[ggerganov]

Sounds good. Do you have an idea how to simulate these synthetic repos with the existing HF model resolution logic?

[ServeurpersoCom]

What about doing it in C++, splitting the existing functions so the resolution takes the file listing as input, then the test just hardcodes in-memory listings

[ServeurpersoCom]

Like this :

diff --git a/common/arg.cpp b/common/arg.cpp
index 28a47eae6..4294ea571 100644
--- a/common/arg.cpp
+++ b/common/arg.cpp
@@ -356,10 +356,7 @@ static bool spec_types_is_default(const common_params & params) {
     return params.speculative.types == std::vector<enum common_speculative_type>{COMMON_SPECULATIVE_TYPE_NONE};
 }

-common_models_handler common_models_handler_init(const common_params & params, llama_example curr_ex) {
-    common_download_hf_plan plan;
-    common_download_hf_plan plan_spec;
-    common_download_hf_plan plan_voc;
+common_download_opts common_models_handler_get_opts(const common_params & params, llama_example curr_ex) {
     common_download_opts opts;

     const bool spec_type_draft_mtp = std::find(params.speculative.types.begin(),
@@ -391,6 +388,16 @@ common_models_handler common_models_handler_init(const common_params & params, l
     opts.download_mmproj = use_mmproj && !params.no_mmproj
                         && params.mmproj.path.empty() && params.mmproj.url.empty();

+    return opts;
+}
+
+common_models_handler common_models_handler_init(const common_params & params, llama_example curr_ex) {
+    common_download_hf_plan plan;
+    common_download_hf_plan plan_spec;
+    common_download_hf_plan plan_voc;
+
+    common_download_opts opts = common_models_handler_get_opts(params, curr_ex);
+
     if (!params.model.hf_repo.empty()) {
         plan = common_download_get_hf_plan(params.model, opts);
     }
@@ -455,15 +462,14 @@ static std::vector<common_download_task> build_url_tasks(const common_params_mod
     return tasks;
 }

-void common_models_handler_apply(common_models_handler & handler, common_params & params, common_download_callback * callback) {
+std::vector<common_download_task> common_models_handler_get_tasks(common_models_handler & handler, common_params & params) {
     std::vector<common_download_task> tasks;

     auto & plan      = handler.plan;
     auto & plan_spec = handler.plan_spec;
     auto & plan_voc  = handler.plan_voc;

-    auto opts = handler.opts; // copy
-    opts.callback = callback;
+    auto & opts = handler.opts;

     // handle plain "url" if needed
     auto handle_url = [&](common_params_model & model) {
@@ -658,6 +664,16 @@ void common_models_handler_apply(common_models_handler & handler, common_params
         });
     }

+    return tasks;
+}
+
+void common_models_handler_apply(common_models_handler & handler, common_params & params, common_download_callback * callback) {
+    auto tasks = common_models_handler_get_tasks(handler, params);
+
+    for (auto & task : tasks) {
+        task.opts.callback = callback;
+    }
+
     // run all tasks in parallel
     if (!params.offline) {
         // if duplicated files are found, only download once (but still call on_done for each task)
diff --git a/common/arg.h b/common/arg.h
index 54a38b9cc..e29fe9fbf 100644
--- a/common/arg.h
+++ b/common/arg.h
@@ -141,6 +141,12 @@ struct common_models_handler {
 // initialize downloading opts and hf_plan if needed, but does not download anything yet
 common_models_handler common_models_handler_init(const common_params & params, llama_example curr_ex);

+// compute the downloading opts from the params (also used by tests)
+common_download_opts common_models_handler_get_opts(const common_params & params, llama_example curr_ex);
+
+// assemble the download tasks and wire the params, without running them (also used by tests)
+std::vector<common_download_task> common_models_handler_get_tasks(common_models_handler & handler, common_params & params);
+
 // check if the model is a preset repo (i.e. has a preset file)
 bool common_models_handler_is_preset_repo(const common_models_handler & handler);

diff --git a/common/download.cpp b/common/download.cpp
index 3776c6c7e..f576a935f 100644
--- a/common/download.cpp
+++ b/common/download.cpp
@@ -723,7 +723,6 @@ static void list_available_gguf_files(const hf_cache::hf_files & files) {
 }

 common_download_hf_plan common_download_get_hf_plan(const common_params_model & model, const common_download_opts & opts) {
-    common_download_hf_plan plan;
     hf_cache::hf_files all;

     auto [repo, tag] = common_download_split_repo_tag(model.hf_repo);
@@ -735,9 +734,17 @@ common_download_hf_plan common_download_get_hf_plan(const common_params_model &
         all = hf_cache::get_cached_files(repo);
     }
     if (all.empty()) {
-        return plan;
+        return {};
     }

+    return common_download_get_hf_plan_from_files(all, model, opts);
+}
+
+common_download_hf_plan common_download_get_hf_plan_from_files(const hf_cache::hf_files & all, const common_params_model & model, const common_download_opts & opts) {
+    common_download_hf_plan plan;
+
+    auto [repo, tag] = common_download_split_repo_tag(model.hf_repo);
+
     // if preset.ini exists in the repo root, download only that file
     for (const auto & f : all) {
         if (f.path == "preset.ini") {
diff --git a/common/download.h b/common/download.h
index 3e789e9e9..cae1564cc 100644
--- a/common/download.h
+++ b/common/download.h
@@ -113,3 +113,6 @@ struct common_download_hf_plan {
     hf_cache::hf_file preset; // if set, only this file is downloaded
 };
 common_download_hf_plan common_download_get_hf_plan(const common_params_model & model, const common_download_opts & opts);
+
+// same resolution with the file listing provided by the caller (also used by tests)
+common_download_hf_plan common_download_get_hf_plan_from_files(const hf_cache::hf_files & all, const common_params_model & model, const common_download_opts & opts);
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index 7a93b19a0..babf2e465 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -258,6 +258,7 @@ llama_build_and_test(test-thread-safety.cpp ARGS -m "${MODEL_DEST}" -ngl 99 -p "
 set_tests_properties(test-thread-safety PROPERTIES FIXTURES_REQUIRED test-download-model)

 llama_build_and_test(test-arg-parser.cpp)
+    llama_build_and_test(test-download-plan.cpp)

 if (NOT LLAMA_SANITIZE_ADDRESS AND NOT GGML_SCHED_NO_REALLOC)
   # TODO: repair known memory leaks

[ServeurpersoCom]

I'm checking if we can run a test that includes the code without modifying it, in order to wrap tests around it.