---

base_model:

• nphearum/PsarAI-2B

tags:

• gguf
• llama.cpp
• unsloth
• gemma4
• psarai
• conversational
• multimodal

---

PsarAI-2B GGUF

GGUF exports for PsarAI-2B, a PsarAI chat model based on unsloth/gemma-4-E2B-it.

The chat template identifies the assistant as PsarAI and uses Gemma's native turn, channel, tool-call, image, audio, and video tokens.

Recommended File

Use Q4_K_M for the best default balance of size, speed, and quality:

llama-cli \
  -hf nphearum/PsarAI-2B-GGUF:Q4_K_M \
  -p "Can AI take human's jobs?"

For higher quality, use Q5_K_M, Q6_K, or Q8_0 if you have enough RAM/VRAM.

llama.cpp Server

llama-server \
  -hf nphearum/PsarAI-2B-GGUF:Q4_K_M \
  --host 0.0.0.0 \
  --port 8080 \
  -c 8192

Then call the OpenAI-compatible endpoint:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "PsarAI-2B",
    "messages": [
      {"role": "user", "content": "Can AI take human's jobs?"}
    ],
    "temperature": 0.7,
    "top_p": 0.9
  }'

Thinking

For normal chatbot use, disable visible thinking in your template/runtime settings when supported:

{"enable_thinking": false}

If thinking is enabled, the template asks the model to keep it short and useful.

Multimodal Projector

This repo includes:

PsarAI-2B.BF16-mmproj.gguf

Use it with a llama.cpp build/runtime that supports Gemma 4 multimodal GGUF. Exact image/audio/video CLI flags may depend on your llama.cpp version.

Files

| File | Size |

|---|---:|

| PsarAI-2B.Q4_K_M.gguf | 3.19 GiB |

| PsarAI-2B.Q5_K_M.gguf | 3.38 GiB |

| PsarAI-2B.Q6_K.gguf | 3.58 GiB |

| PsarAI-2B.Q8_0.gguf | 4.63 GiB |

| PsarAI-2B.Q4_K_S.gguf | 3.13 GiB |

| PsarAI-2B.Q3_K_M.gguf | 2.98 GiB |

| PsarAI-2B.IQ4_XS.gguf | 3.08 GiB |

| PsarAI-2B.IQ4_NL.gguf | 3.14 GiB |

| PsarAI-2B.IQ3_M.gguf | 2.14 GiB |

| PsarAI-2B.F16.gguf | 8.67 GiB |

| PsarAI-2B.BF16-mmproj.gguf | 0.92 GiB |

Quantization Guide

• Q4_K_M: recommended default
• Q5_K_M: better quality with moderate extra size
• Q6_K: strong quality if memory is available
• Q8_0: near full precision, largest practical runtime file
• Q3_ / IQ3_: smaller files, lower quality

Notes

• Context length in the source config is up to 131072 tokens, but practical context depends on your runtime memory.
• This model uses a Gemma-style chat template, not Qwen XML-style tool calls.
• If you upload only a subset of files, update the file table above to match the repo contents.