---

pretty_name: VibeThinker-1.5B GGUF

author: siruncahined

license: apache-2.0

tags:

• qwen2
• qwen
• gguf
• quantized
• VibeThinker
• Math
• 1.5B
• text-generation
• q4_k_m
• q5_k_m
• q6_k
• q8_0

widget:

• text: Tell me about the VibeThinker-1.5B model.

---

VibeThinker-1.5B GGUF

This is a quantized version of the WeiboAI/VibeThinker-1.5B model, converted to GGUF format for use with llama.cpp and compatible tools.

Quantization Details

The model was quantized using llama.cpp to the following formats:

• FP16: The original model was converted to FP16 GGUF as an intermediate step.
• Q4_K_M: Quantized using the Q4_K_M method.
• Q5_K_M: Quantized using the Q5_K_M method.
• Q6_K: Quantized using the Q6_K method.
• Q8_0: Quantized using the Q8_0 method.

Original Model Card Summary:

• Model ID: WeiboAI/VibeThinker-1.5B
• Original Repository: https://huggingface.co/WeiboAI/VibeThinker-1.5B

How to Use

You can use these GGUF files with llama.cpp or other tools that support the GGUF format. Download the desired quantization level and use it with your llama.cpp compatible inference engine.

Example usage with llama.cpp (replace [quant_level] with your desired quantization, e.g., q4_k_m):

./main -m VibeThinker-1.5B-[quant_level].gguf -p "Hello, what is your name?" -n 128

Files Provided

• VibeThinker-1.5B-f16.gguf (FP16)
• VibeThinker-1.5B-q4_k_m.gguf (Q4_K_M)
• VibeThinker-1.5B-q5_k_m.gguf (Q5_K_M)
• VibeThinker-1.5B-q6_k.gguf (Q6_K)
• VibeThinker-1.5B-q8_0.gguf (Q8_0)