---

language:

• multilingual

license: mit

library_name: gguf

tags:

• gguf
• llama.cpp
• phi
• phi-4
• microsoft
• quantized
• q4_k_m
• text-generation
• inference

base_model: microsoft/Phi-4-mini-instruct

pipeline_tag: text-generation

---

Phi-4-mini-instruct-Q4_K_M-GGUF

Model Overview

This repository contains a Q4_K_M GGUF quantization of microsoft/Phi-4-mini-instruct.

The model was converted to the GGUF format and quantized using the Q4_K_M quantization method. No additional training, fine-tuning, alignment, or modifications were performed.

Base Model

• Original model: "microsoft/Phi-4-mini-instruct"
• Format: GGUF
• Quantization: Q4_K_M
• Training: None (conversion and quantization only)

Intended Use

This model is intended for:

• Local inference
• Resource-efficient deployment
• Experimentation
• GGUF-compatible inference engines such as "llama.cpp"

Model Files

| File | Description |

|-------------|-------------|

| "Phi-4-mini-instruct-Q4_K_M.gguf" | GGUF model quantized using the Q4_K_M method |

Quantization

This repository contains only one quantized version:

• Q4_K_M

Q4_K_M provides a good balance between model quality, inference speed, and memory usage, making it a popular choice for consumer hardware.

Compatibility

The model is compatible with software supporting the GGUF format, including:

• llama.cpp
• LM Studio
• Jan
• KoboldCpp
• Ollama (after model import)
• Other GGUF-compatible runtimes

Limitations

This repository does not introduce any changes to the original model beyond GGUF conversion and quantization.

All capabilities, strengths, weaknesses, and safety characteristics are inherited from the original microsoft/Phi-4-mini-instruct model.

As with all large language models, outputs may contain inaccuracies, hallucinations, or biased content.

License

Please refer to the license and usage terms of the original microsoft/Phi-4-mini-instruct model before using or redistributing this quantized version.

Credits

• Original model: Microsoft
• GGUF conversion and Q4_K_M quantization: VladimirRH