Phi-4 GGUF

Quantized GGUF versions of microsoft/Phi-4 generated using llama.cpp on an Oracle Cloud VM.

Repositories & Quantization Details

This repository contains the following quantization formats:

| Quantization Type | File Size | Quantization Duration | Peak RAM Used | Description |

| ----------------- | --------- | --------------------- | ------------- | ----------- |

| Q4_K_M | 8.44 GB | 145.9s | 31911.6 MB | Medium-sized, good balance of size/perplexity. |

| Q5_K_M | 9.78 GB | 119.4s | 31911.6 MB | Medium-sized, good balance of size/perplexity. |

| Q6_K | 11.20 GB | 146.1s | 31911.6 MB | Medium-sized, good balance of size/perplexity. |

| Q8_0 | 14.51 GB | 120.2s | 31911.3 MB | Medium-sized, good balance of size/perplexity. |

Sanity Check Prompts & Responses

Here are the responses generated by the quantized models to verify coherent text generation.

Format: Q4_K_M

• Prompt: What is the capital of France? Answer in one word.
• Response:

```text

What is the capital of France? Answer in one word. [end of text]

```

• Prompt: Write a short four-line poem about coding in Python.
• Response:

```text

Write a short four-line poem about coding in Python. The poem should rhyme and mention the concepts of 'variables', 'loops', and 'functions'. [end of text]

```

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

```text

Explain why the sky is blue in one simple sentence. [end of text]

```

Format: Q5_K_M

• Prompt: What is the capital of France? Answer in one word.
• Response:

```text

What is the capital of France? Answer in one word. [end of text]

```

• Prompt: Write a short four-line poem about coding in Python.
• Response:

```text

Write a short four-line poem about coding in Python. [end of text]

```

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

```text

Explain why the sky is blue in one simple sentence. [end of text]

```

Format: Q6_K

• Prompt: What is the capital of France? Answer in one word.
• Response:

```text

What is the capital of France? Answer in one word. [end of text]

```

• Prompt: Write a short four-line poem about coding in Python.
• Response:

```text

Write a short four-line poem about coding in Python. The poem must rhyme and include the words "syntax" and "loop."

Answer

In Python's realm where the syntax is neat,

A loop spins round, a rhythmic feat.

With each command, the code does swoop,

In this dance of logic, we find our loop. [end of text]

```

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

```text

Explain why the sky is blue in one simple sentence. [end of text]

```

Format: Q8_0

• Prompt: What is the capital of France? Answer in one word.
• Response:

```text

What is the capital of France? Answer in one word. FAILURE strictly strictlyibilibil Pir Sanfordbiratern Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur(exec Bur Bur Bur Bur Bur Bur Bur

```

• Prompt: Write a short four-line poem about coding in Python.
• Response:

```text

Write a short four-line poem about coding in Python.nymnymnymnymnym-docnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnym Grosnym^K^K Roths Grosnym^K Gros^KCelekö^K Roths Roths Roths径 Roths '\" Roths Roths Roths Roths '\" Roths Roths Roths Roths Roths^K Roths Roths Roths

```

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

```text

Explain why the sky is blue in one simple sentence.Teacher " CwikAimage-in-inur-forururur-forurururururvaulturururvaultvaultvaultvault_codegenvault239239239239239239239239239239239 gh239239239239239urd239

```

Pipeline Logs

• Base GGUF Conversion Peak RAM: 4080.6 MB, Duration: 294.2s.
• Logged at: Fri Jun 19 18:11:27 2026