---

language:

• en
• zh

license: apache-2.0

base_model: ewinregirgojr/Qwen3.8-14B-Instruct-Turbo

tags:

• gguf
• llama.cpp
• qwen
• qwen3
• 14b
• deltanet
• linear-attention
• hybrid-attention
• distillation
• pruned
• reasoning
• tool-calling
• agent
• coding

pipeline_tag: text-generation

inference: false

---

STABLE VERSION (https://huggingface.co/ewinregirgojr/Qwen3.8-19B-Pruned)

Qwen3.8-14B-Instruct-Turbo-GGUF

Official GGUF quantizations of Qwen3.8-14B-Instruct-Turbo, a 14.2B parameter pruned and distilled hybrid reasoning and agent model derived from Qwen3.8-27B.

This repository provides high-performance quantized .gguf binaries optimized for local inference across CPU, Apple Silicon (Metal), and NVIDIA CUDA hardware via llama.cpp, Ollama, LM Studio, and Text-Generation-WebUI.

---

📦 Provided GGUF Files

| Filename | Quant Method | File Size | Recommended Hardware / VRAM | Use Case |

| :--- | :---: | :---: | :--- | :--- |

| Qwen3.8-14B-Instruct-Turbo-Q4_K_M.gguf | Q4_K_M | 9.16 GB | 8GB - 12GB VRAM / 16GB RAM | Recommended default. Optimal balance of speed, low memory, and high generation quality. |

| Qwen3.8-14B-Instruct-Turbo-Q5_K_M.gguf | Q5_K_M | 10.60 GB | 12GB - 16GB VRAM / 24GB RAM | High-precision quantization for complex coding and chained multi-step reasoning. |

| Qwen3.8-14B-Instruct-Turbo-Q8_0.gguf | Q8_0 | 14.58 GB | 16GB - 24GB VRAM / 32GB RAM | Near-lossless master precision matching FP16 quality. |

---

📊 Empirical Benchmark Evaluation Results

The model was evaluated using greedy decoding on NVIDIA T4 GPU hardware across official standardized benchmark splits:

| Benchmark Capability | Evaluation Dataset / Metric | Evaluated Count | Qwen3.8-14B-Turbo (Empirical) | Qwen3.8-27B (Baseline) | Evaluation Protocol |

| :--- | :--- | :---: | :---: | :---: | :--- |

| Scientific Reasoning | GPQA Diamond | 50 Questions | 70.0% (35 / 50) | 89.2% | Exact-match multiple choice on expert graduate-level science problems. |

| Agent / Function Calling | BFCL / XLAM Tool Use | 50 Test Cases | 22.0% (11 / 50) | 67.1% | Strict AST argument parsing and function dispatch validation. |

| Coding & Program Synthesis | HumanEval / LiveCode | 6 Evaluation Sets | 7.56 – 7.60 tok/s | ~5.5 tok/s (FP16) | Measured greedy decoding throughput on NVIDIA T4 GPU. |

| Knowledge Distillation | Multi-Turn Curricula | 200 Iterations | -82.58% Loss Drop | N/A | Loss dropped from 4.7109 to 0.8207 ($r=32, lpha=64$). |

---

🚀 Quickstart & Usage

1. Using with llama.cpp CLI

Run inference directly on your GPU using llama-cli:

# Run with Q4_K_M offloaded to CUDA GPU
llama-cli -m Qwen3.8-14B-Instruct-Turbo-Q4_K_M.gguf \
  -ngl 99 \
  -c 8192 \
  --temp 0.2 \
  -p "<|im_start|>system\nYou are an expert assistant.<|im_end|>\n<|im_start|>user\nWrite a Python function for prefix trie search.<|im_end|>\n<|im_start|>assistant\n<think>\n"

2. Using with llama-server (OpenAI-Compatible API)

Start a local OpenAI-compatible API server on port 8080:

llama-server -m Qwen3.8-14B-Instruct-Turbo-Q4_K_M.gguf \
  -ngl 99 \
  -c 8192 \
  --port 8080 \
  --host 0.0.0.0

You can now connect any OpenAI-compatible client (e.g. LangChain, AutoGen, Open-WebUI) to http://localhost:8080/v1.

3. Using with Ollama

Create a Modelfile:

FROM ./Qwen3.8-14B-Instruct-Turbo-Q4_K_M.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
<think>
{{ .Response }}"""
PARAMETER temperature 0.2
PARAMETER top_p 0.9
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|im_start|>"

Then build and run:

ollama create qwen3.8-14b-turbo -f Modelfile
ollama run qwen3.8-14b-turbo

---

📜 Citation & Attribution

@misc{qwen3.8_14b_instruct_turbo_gguf_2026,
  author = {ewinregirgojr},
  title = {Qwen3.8-14B-Instruct-Turbo-GGUF: Optimized 14.2B Hybrid Linear-Attention Reasoning Model},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ewinregirgojr/Qwen3.8-14B-Instruct-Turbo-GGUF}}
}