---

license: llama3.2

base_model: meta-llama/Llama-3.2-1B-Instruct

tags:

• gguf
• llama
• llama-3.2
• quantized
• llama.cpp
• q4_k_m

pipeline_tag: text-generation

---

Llama-3.2-1B-Instruct — GGUF

Built with Llama.

A Q4_K_M GGUF quantization of Meta's

Llama-3.2-1B-Instruct,

for use with llama.cpp and compatible

runtimes (Ollama, LM Studio, llama-cpp-python, etc.).

Q4_K_M is the sweet spot for a 1B model — smallest footprint with still-solid

quality.

| File | Quant | Size | Notes |

|------|-------|------|-------|

| Llama-3.2-1B-Instruct-Q4_K_M.gguf | Q4_K_M | ~770 MB | Smallest, good quality |

Usage

llama.cpp

llama-cli -hf andreolf/Llama-3.2-1B-Instruct-GGUF:Q4_K_M -p "Hello!"

Ollama

ollama run hf.co/andreolf/Llama-3.2-1B-Instruct-GGUF:Q4_K_M

llama-cpp-python

from llama_cpp import Llama
llm = Llama.from_pretrained(
    repo_id="andreolf/Llama-3.2-1B-Instruct-GGUF",
    filename="Llama-3.2-1B-Instruct-Q4_K_M.gguf",
    n_ctx=4096,
)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "Hello!"}]
)["choices"][0]["message"]["content"])

How it was made

Converted to an F16 GGUF with convert_hf_to_gguf.py, then quantized to Q4_K_M

with llama-quantize from llama.cpp. No

fine-tuning or modification was applied to the weights.

License & attribution

This is a quantized redistribution of Meta's Llama-3.2-1B-Instruct and is

governed by the Llama 3.2 Community License.

Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta

Platforms, Inc. All Rights Reserved. All credit for the underlying model goes to

Meta; this repo only provides a GGUF conversion for convenience.