0

When converting mistralai/Mistral-Small-3.2-24B-Instruct-2506 to GGUF (via llama_cpp), I get an error saying the tokenizer.json file is missing. After re-examining the HF repo, there in fact, is not a tokenizer.json file in the repo; although there is one for the other models I have converted, and the base model.

However, others have successfully converted and quantised this model; which is my endgame. What do I need to do to convert this model to GGUF format?

Can I use the tokenizer.json file from the base model?

What I have done so far:

Posting here is my last resort. Over the last week I have done the following, with no response:

  • Searched the llama.cpp GitHub issues page for related issues; no solution found for this specific case.
  • Posted on two HuggingFace communities, one for MistalAI (the author of the model) and another on a member who has successfully converted the model to GGUF. No responses.
  • Directly emailed a member who has successfully converted the model to GGUF. No response.
  • Posted a question on AI Stack Exchange (and got my hand slapped for being off-topic, as it's not a theoretical question).
Improve this question
Related questions
1
Fine-tuned Qwen2.5-7B model loops infinitely in Ollama but works fine with transformers
2
Convert safetensors model format(LLaVA model) into gguf format
2
How to quantize a HF safetensors model and save it to llama.cpp GGUF format with less than q8_0 quantization?
Load 4 more related questions Show fewer related questions

0

Reset to default

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.