hotdogs/Agents-A1-4B-Fable-Preview-GGUF overview

---

license: agpl-3.0

language:

- en

- th

tags:

- agents

- agents-a1

- fable

- gguf

- vision

- image-text-to-text

- vlm

- sft

- reasoning

- tool-call

- tool-use

- function-calling

- chatml

- qwen

- preview

base_model:

- hotdogs/Agents-A1-4B-Fable-Preview

datasets:

- hotdogs/uka-fable-reasoning

library_name: gguf

pipeline_tag: image-text-to-text

---

<h1 align="center">🤖 Agents-A1-4B-Fable-Preview-GGUF</h1>

<p align="center">

<b>GGUF Quantized — 4B Vision-Language Agent Model · Fable Reasoning · Tool-Calling</b>

</p>

<p align="center">

<img src="https://img.shields.io/badge/license-AGPL--3.0-red">

<img src="https://img.shields.io/badge/Base-hotdogs/Agents--A1--4B--Fable--Preview-blue">

<img src="https://img.shields.io/badge/GGUF-Q4_K_M-brightgreen">

<img src="https://img.shields.io/badge/Vision-✅-brightgreen">

<img src="https://img.shields.io/badge/Fable-Reasoning-orange">

</p>

<br>

> GGUF quantized version of hotdogs/Agents-A1-4B-Fable-Preview — optimized for llama.cpp inference with vision support.

---

✨ Key Features

| Capability | Description |

|------------|-------------|

| 🖼️ Vision Understanding | Image-text-to-text with mmproj |

| 🧠 Fable Reasoning | Step-by-step CoT with <think> blocks |

| 🔧 Tool Calling | llama.cpp --tools all support |

| 💬 Multi-turn | Trained on full agent trajectories |

| 🌏 Thai + English | Native bilingual support |

| 💻 Code & Shell | Python, bash, system tasks |

| ⚡ Fast Inference | IQ4_NL fits in ~3 GB VRAM |

---

📦 Downloads

| File | Size | Description |

|------|:----:|-------------|

| Agents-A1-4B-Fable-IQ4_NL.gguf | 2.61 GB | Recommended — best quality/speed balance for 8GB VRAM |

| Agents-A1-4B-Fable-Q4_K_M_imatrix.gguf | 2.71 GB | Q4_K_M + imatrix — slightly higher quality |

| Agents-A1-4B-Fable-Q6_K_imatrix.gguf | 3.46 GB | Q6_K + imatrix — higher quality, more VRAM |

| Agents-A1-4B-Fable-Q8_0_imatrix.gguf | 4.48 GB | Q8_0 + imatrix — almost lossless |

| Agents-A1-4B-Fable-f16.gguf | 8.42 GB | Full BF16 precision |

| Agents-A1-4B-mmproj.gguf | 672 MB | Vision projector for image understanding |

| imatrix.dat | 3.63 MB | Importance matrix data |

> 🎯 IQ4_NL is recommended for 8GB VRAM users — fits comfortably even at 128K context with flash-attention.

---

🚀 Usage

Docker (Recommended)

sudo docker run --rm -p 8080:8080 \
  -v /root/models/:/models \
  --gpus all \
  --ulimit memlock=-1:-1 \
  --env CUDA_VISIBLE_DEVICES=0 \
  ghcr.io/ggml-org/llama.cpp:full-cuda --server \
  -m /models/Agents-A1-4B-Fable-IQ4_NL.gguf \
  --mmproj /models/Agents-A1-4B-mmproj.gguf \
  --host 0.0.0.0 --port 8080 \
  --n-gpu-layers 999 \
  --ctx-size 131072 \
  --batch-size 4096 \
  --ubatch-size 256 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --flash-attn on \
  --cont-batching \
  --mlock \
  --temp 0.95 \
  --top-k 40 \
  --top-p 0.9 \
  --min-p 0.0 \
  -n -1 \
  --no-mmap \
  --parallel 1 --tools all \
  --dry-multiplier 0.05 \
  --jinja --dry-sequence-breaker none \
  --repeat-penalty 1.1

Parameter Explanation

| Parameter | Purpose |

|-----------|---------|

| --mmproj | Vision projector for image understanding |

| --ctx-size 131072 | 128K context window |

| --flash-attn on | Flash attention for speed |

| --cache-type-k/v f16 | BF16 KV cache for quality |

| --cont-batching | Continuous batching for multi-turn |

| --tools all | Enable tool/function calling |

| --jinja | Use Jinja2 chat template |

| --mlock | Lock memory for performance |

llama.cpp (Direct)

# Quick text-only test
./llama-cli -m Agents-A1-4B-Fable-IQ4_NL.gguf \
  -p "Hello" -n 100 --temp 0.6 -ngl 999

# Vision inference
./llama-cli -m Agents-A1-4B-Fable-IQ4_NL.gguf \
  --mmproj Agents-A1-4B-mmproj.gguf \
  --image photo.jpg \
  -p "What is in this image?" -n 256 --temp 0.6 -ngl 999

---

🧬 Model Information

This is a GGUF quantized version of hotdogs/Agents-A1-4B-Fable-Preview, which is a fine-tune of InternScience/Agents-A1-4B.

| Parameter | Value |

|-----------|:-----:|

| Base Model | hotdogs/Agents-A1-4B-Fable-Preview |

| Parameters | ~4.29B |

| Architecture | Qwen3.5 hybrid (Linear + Full attention) |

| Vision | ✅ 24-layer ViT encoder via mmproj |

| Context | Up to 128K tokens |

| Format | ChatML (Jinja2 template) |

| Fine-tuning | Fable-style reasoning traces (3,500 samples, 3 epochs) |

---

🙏 Acknowledgements / ขอบคุณ

• InternScience — For the Agents-A1-4B base model and mmproj vision projector 🙏
• mmproj source — Extracted from InternScience/Agents-A1-4B-Q4_K_M-GGUF
• Qwen Team (Alibaba) — For the Qwen3.5 architecture
• Unsloth AI — For training optimizations
• All dataset contributors and the open-source AI community ❤️

---

💖 Support / โปรดสนับสนุน

If you find this model useful, please consider supporting my work!

หากคุณคิดว่าโมเดลนี้มีประโยชน์ กรุณาสนับสนุนผลงานของฉันด้วยนะคะ! 🙏

<p align="center">

<img src="https://huggingface.co/hotdogs/Qwen35B-Agent-R2/raw/main/donate.webp" alt="Bitcoin QR — Donate" width="256">

</p>

₿ Bitcoin — BTC:

bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v

Thank you for your support! 🙏✨

ขอบคุณมากๆ สำหรับการสนับสนุนค่า! 💖🤗

---

Built with ❤️ by UKA — 18-year-old coder & cybersecurity expert