---

license: bigcode-openrail-m

library_name: transformers

pipeline_tag: text-generation

base_model: bigcode/starcoder2-3b

tags:

• gguf
• llama.cpp
• text-generation
• code
• code-completion
• starcoder2
• 3b
• quantization
• q4_k_m

---

StarCoder2-3B GGUF

<div align="center">

<img alt="Model" src="https://img.shields.io/badge/model-StarCoder2--3B-8A2BE2?style=for-the-badge">

<img alt="Published format" src="https://img.shields.io/badge/GGUF-Q4_K_M-FFD21E?style=for-the-badge">

<img alt="Task" src="https://img.shields.io/badge/task-code--completion-00A6A6?style=for-the-badge">

<img alt="Parameters" src="https://img.shields.io/badge/params-3B-16A34A?style=for-the-badge">

<img alt="Provenance" src="https://img.shields.io/badge/weights-community%20Q4%20conversion-E8590C?style=for-the-badge">

<img alt="License" src="https://img.shields.io/badge/license-OpenRAIL--M-7C3AED?style=for-the-badge">

</div>

Source model · HF release · Build hub

This release contains a validated Q4_K_M artifact for the StarCoder2-3B base model. The weights come from the established second-state/StarCoder2-3B-GGUF Q4_K_M conversion; no weight values were changed. The GGUF metadata was repaired to add the required tokenizer.ggml.pre=starcoder field, which removes llama.cpp's missing-pre-tokenizer warning.

Format status

| File | Status | Purpose |

|---|---|---|

| starcoder2-3b-Q4_K_M.gguf | Published | Compact code model |

| starcoder2-3b-F16.gguf | Not published | Source checkpoint download exceeded the available transfer budget |

| starcoder2-3b-Q8_0.gguf | Not published | Same reason; it would have to be requantised from the source, not from Q4 |

The source checkpoint is 12.1 GB, so a fresh source conversion was not repeated here. The included build_gguf.py can build F16, Q8_0, and Q4_K_M from the pinned source when the download and disk requirements are available.

Verified code completions

Verbatim starcoder2-3b-Q4_K_M.gguf completions, --temp 0, 24 new tokens. The prompt is the def/for/SELECT line plus the already-given return or print(, and the model's text is reproduced as-is.

| Prompt | Model completion |

|---|---|

| def is_even(number):<br> return | number % 2 == 0 |

| def reverse_text(text):<br> return | text[::-1] |

| def factorial(number):<br> if number <= 1:<br> return | 1<br> else:<br> return number * factorial(number - 1) |

| def sum_list(numbers):<br> return | sum(numbers) |

| def find_max(numbers):<br> return | max(numbers) |

| for index, item in enumerate(items):<br> print( | index, item) |

Python is clean. SQL is not, and it fails differently from DeepSeek-Coder: SELECT name FROM users WHERE active = drifts into a JavaScript then(result => ...) block, and SELECT COUNT(*) FROM orders WHERE total > emits 10000000000000000000000. StarCoder2-3B is a base model, so the trailing print(...) test calls and generated helper functions are trimmed above rather than presented as intended output.

Validation

WikiText-2 raw test evaluation, 8 chunks of 512 tokens:

| Format | PPL |

|---|---:|

| Q4_K_M, metadata repaired | 11.7857 +/- 0.67032 |

A deterministic code-completion smoke test completed successfully:

def add(a, b):
    return a + b

Build

python -m pip install -r requirements-build.txt
python build_gguf.py --model-id bigcode/starcoder2-3b

The builder pins llama.cpp commit 6b790a9c291b5d7af3312bbf9f0c558aa023b13e and the upstream model revision. It does not upload or overwrite this repository.

License and provenance

BigCode OpenRAIL-M. See the upstream model card, the LICENSE notice, and the full license agreement linked there. The Q4_K_M weights are attributed to second-state/StarCoder2-3B-GGUF; this repository only repairs missing GGUF tokenizer metadata and adds documentation.