---

license: llama3.2

base_model: OctoThinker/OctoThinker-8B-Long-Base

tags:

- llama

- gguf

- quantized

- reasoning

- math

language:

- en

pipeline_tag: text-generation

library_name: gguf

---

OctoThinker-8B-Long-Base GGUF

GGUF quantizations of OctoThinker/OctoThinker-8B-Long-Base.

OctoThinker-8B-Long-Base is a mid-trained language model built on the Llama-3 family, designed to create a reinforcement learning–friendly base language model. It is trained using the Stable-then-Decay two-stage mid-training paradigm on large-scale mathematical datasets (MegaMath-Web-Pro-Max) and diversified Long Chain-of-Thought data, ensuring rapid convergence and improved RL stability.

For more details, see the paper.

---

Available Quantizations

| File | Bits | Size (approx) | Use Case |

|------|------|---------------|----------|

| model_f16.gguf | 16 | ~16.1 GB | Maximum quality, reference |

| model_q8_0.gguf | 8 | ~8.5 GB | Best quality, if VRAM allows |

| model_q6_k.gguf | 6 | ~6.6 GB | Great quality |

| model_q5_k_m.gguf | 5 | ~5.7 GB | Balanced quality/size |

| model_q5_k_s.gguf | 5 | ~5.5 GB | Slightly smaller Q5 |

| model_q4_k_m.gguf | 4 | ~4.8 GB | Good quality, recommended |

| model_q4_k_s.gguf | 4 | ~4.6 GB | Smaller Q4 |

| model_q3_k_l.gguf | 3 | ~4.0 GB | Low VRAM, decent quality |

| model_q3_k_m.gguf | 3 | ~3.7 GB | Lower VRAM |

| model_q3_k_s.gguf | 3 | ~3.5 GB | Minimum Q3 |

| model_q2_k.gguf | 2 | ~3.0 GB | Very low VRAM, reduced quality |

---

VRAM Requirements

| Quantization | VRAM (approx) |

|--------------|---------------|

| F16 | ~17 GB |

| Q8_0 | ~9.5 GB |

| Q6_K | ~7.5 GB |

| Q5_K_M | ~6.5 GB |

| Q4_K_M | ~5.5 GB |

| Q3_K_M | ~4.5 GB |

| Q2_K | ~3.5 GB |

---

Usage

llama.cpp

./llama-cli -m model_q4_k_m.gguf -p "Your prompt here" -n 512

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="model_q4_k_m.gguf", n_ctx=8192)
output = llm("Your prompt here", max_tokens=512)
print(output["choices"][0]["text"])

LM Studio

Download the desired .gguf file and load it directly in LM Studio.

Ollama

ollama run hf.co/tinyopsec/OctoThinker-8B-Long-Base-GGUF

---

Citation

@article{wang2025octothinker,
  title={OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling},
  author={Wang, Zengzhi and Zhou, Fan and Li, Xuefeng and Liu, Pengfei},
  year={2025},
  journal={arXiv preprint arXiv:2506.20512},
  note={Preprint}
}