---

license: mit

library_name: gguf

tags:

• gguf
• llama.cpp
• phi-4
• phi-4-mini
• reasoning
• microsoft
• quantized
• local-inference

pipeline_tag: text-generation

---

Phi-4-mini-reasoning GGUF

GGUF conversion and Q4_K_M quantization of

Microsoft Phi-4-mini-reasoning for efficient local inference.

Original Model

Official model:

https://huggingface.co/microsoft/Phi-4-mini-reasoning

This repository contains community-created GGUF conversions of the

original model.

Please refer to the official model card for complete information

about the model, training, intended use, limitations, and license.

Files

| File | Format | Approx. Size | Use |

|---|---|---:|---|

| Phi-4-mini-reasoning-F16.gguf | F16 | ~7.2 GB | Higher precision |

| Phi-4-mini-reasoning-Q4_K_M.gguf | Q4_K_M | ~2.4 GB | Recommended |

Recommended Version

Q4_K_M

Phi-4-mini-reasoning-Q4_K_M.gguf is recommended for most local

inference setups.

It provides a much smaller memory footprint than the F16 version while

maintaining a good balance between quality and efficiency.

F16

The F16 version is provided for users who have sufficient memory and

want to use a higher-precision GGUF representation.

Quantization

Conversion pipeline:

Microsoft Phi-4-mini-reasoning

→ F16 GGUF

→ Q4_K_M

The Q4_K_M file was generated using the quantization tools from

llama.cpp.

Quantization:

Q4_K_M

No importance matrix was used.

llama.cpp

Basic inference:

llama-cli -m Phi-4-mini-reasoning-Q4_K_M.gguf

GPU offloading:

llama-cli -m Phi-4-mini-reasoning-Q4_K_M.gguf -ngl 99

Run a local server:

llama-server -m Phi-4-mini-reasoning-Q4_K_M.gguf -ngl 99

Compatible Software

The GGUF files can be used with software supporting the GGUF format,

including:

• llama.cpp
• llama-cpp-python
• LM Studio
• Jan
• Other GGUF-compatible inference engines

Context

Phi-4-mini-reasoning is designed for reasoning-focused language

generation.

For model-specific capabilities and limitations, refer to the official

Microsoft model card.

License

The original model is provided under its respective Microsoft license.

Please review the official model repository before using or

redistributing the model:

https://huggingface.co/microsoft/Phi-4-mini-reasoning

Disclaimer

This is a community GGUF conversion and quantization.

This repository is not an official Microsoft repository.

Conversion Environment

Conversion and quantization were performed using llama.cpp.

Conversion:

convert_hf_to_gguf.py

Quantization:

llama-quantize

Quantization type:

Q4_K_M