---

base_model: modrill/CodeThink-V4-Qwen3-8B

library_name: transformers

pipeline_tag: text-generation

license: apache-2.0

language:

• en

tags:

• gguf
• llama-cpp
• text-generation
• code
• reasoning
• qwen3
• livecodebench

---

CodeThink-V4-Qwen3-8B GGUF

This repository provides GGUF quantizations for modrill/CodeThink-V4-Qwen3-8B.

CodeThink-V4-Qwen3-8B is an 8.19B-parameter code reasoning research model built on Qwen/Qwen3-8B-Base. It was trained using V4 LoRA SFT distillation on paired thinking traces from Qwen/Qwen3-30B-A3B-Thinking-2507, then merged to full weights. The model is tuned for programmatic problem solving, algorithmic code generation, and step-by-step reasoning inside <think> blocks.

All GGUF files were converted from the original merged safetensors weights using llama.cpp at native BF16 precision, then quantized into standard k-quant variants.

---

Available Files and Quantizations

| File | Quant Type | Size | Description / Recommendation |

| :--- | :--- | :--- | :--- |

| CodeThink-V4-Qwen3-8B-BF16.gguf | BF16 | 15.26 GB | Full precision base conversion. Highest fidelity reference weights. |

| CodeThink-V4-Qwen3-8B-Q8_0.gguf | Q8_0 | 8.11 GB | Near-lossless 8-bit quantization. Recommended for best code generation accuracy. |

| CodeThink-V4-Qwen3-8B-Q6_K.gguf | Q6_K | 6.26 GB | High quality retention with minimal degradation. Excellent balance. |

| CodeThink-V4-Qwen3-8B-Q5_K_M.gguf | Q5_K_M | 5.45 GB | Solid balance between memory usage and code syntax adherence. |

| CodeThink-V4-Qwen3-8B-Q4_K_M.gguf | Q4_K_M | 4.68 GB | Fast, lightweight 4-bit quant. Recommended default for local workstation inference. |

| CodeThink-V4-Qwen3-8B-Q3_K_M.gguf | Q3_K_M | 3.84 GB | Compact footprint when GPU or system memory headroom is strictly limited. |

| CodeThink-V4-Qwen3-8B-Q2_K.gguf | Q2_K | 3.06 GB | Maximum compression for testing on memory-limited hardware. |

---

Technical Specifications

• Base Architecture: Qwen3ForCausalLM (model_type: qwen3)
• Parameters: ~8.19B
• Layers: 36
• Hidden Size: 4096
• Intermediate Size: 12288
• Attention Heads: 32 query heads, 8 key/value heads (Grouped Query Attention)
• Head Dimension: 128
• Context Window: 32,768 tokens
• Vocabulary Size: 151,936 tokens
• Native Precision: bfloat16
• License: Apache 2.0

---

Benchmark Results (DEV256)

Evaluated on the 256-problem LiveCodeBench-derived development split (seed 3407, thinking mode, without <think> prefill, max generation ~32k tokens, pass@1):

| Model | pass@1 | Cap (hit 32k tokens) | Notes |

| :--- | :---: | :---: | :--- |

| CodeThink-V4-Qwen3-8B | 81 / 256 | 130 | V4 thinking contract (step-000921 endpoint) |

| Qwen3-8B-Base | 55 / 256 | -- | Historical base reference |

---

Prompt Format

CodeThink-V4-Qwen3-8B uses ChatML markup. To trigger reasoning traces, prompt the assistant turn and allow the model to emit its own <think> block (do not prefill <think> manually):

<|im_start|>system
You are an expert Python programmer. You will be given a question (problem specification) and will generate a correct Python program that matches the specification and passes all tests. You will NOT return anything except for the program.<|im_end|>
<|im_start|>user
Write a Python function to solve the two-sum problem efficiently.<|im_end|>
<|im_start|>assistant

---

How to Use

1. With llama.cpp

Run interactive generation with llama-cli:

llama-cli -m ./CodeThink-V4-Qwen3-8B-Q4_K_M.gguf \
    -p "<|im_start|>system\nYou are an expert Python programmer.<|im_end|>\n<|im_start|>user\nWrite a Python function to check if a string is a palindrome.<|im_end|>\n<|im_start|>assistant\n" \
    -n 1024 \
    -c 4096 \
    --temp 0.6 \
    --top-p 0.95 \
    --top-k 20

Launch an OpenAI-compatible local API server:

llama-server -m ./CodeThink-V4-Qwen3-8B-Q4_K_M.gguf \
    --host 127.0.0.1 \
    --port 8080 \
    -c 8192

2. With Ollama

Create a file named Modelfile in the same directory:

FROM ./CodeThink-V4-Qwen3-8B-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20

Build and run the model in Ollama:

ollama create codethink-8b -f Modelfile
ollama run codethink-8b

3. With LM Studio

1. Copy the desired .gguf file into your local LM Studio models directory.
2. Select CodeThink-V4-Qwen3-8B in the model selector.
3. Choose ChatML as the preset prompt template.

---

Attribution & Credits

• Base weights and architecture by Qwen Team.
• Fine-tuning and distillation by modrill.
• GGUF conversions created by Pink Pixel.