FalconSuzuki/Llama-3.1-Swallow-8B-Instruct-v0.5-Q4_K_M-GGUF overview
Llama 3.1 Swallow 8B Instruct v0.5 Q4 K M GGUF This is a Q4 K M GGUF quantization of tokyotech llm/Llama 3.1 Swallow 8B Instruct v0.5 https://huggingface.co/to…
Runs locally from ~4.58 GB disk (8 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| llama-3.1-swallow-8b-instruct-v0.5-Q4_K_M.gguf | GGUF | Q4_K_M | 4.58 GB | Download |
Model Details
| Model ID | FalconSuzuki/Llama-3.1-Swallow-8B-Instruct-v0.5-Q4_K_M-GGUF |
|---|---|
| Author | FalconSuzuki |
| Pipeline | — |
| License | llama3.3 |
| Base model | tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5 |
| Last modified | 2026-07-30T11:59:56.000Z |
Model README
---
license: llama3.3
base_model: tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5
---
Llama-3.1-Swallow-8B-Instruct-v0.5-Q4_K_M-GGUF
This is a Q4_K_M GGUF quantization of tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5, converted using llama.cpp.
Usage with Ollama
ollama run hf.co/FalconSuzuki/Llama-3.1-Swallow-8B-Instruct-v0.5-Q4_K_M-GGUF:Q4_K_M
Usage with llama.cpp
llama-cli -m llama-3.1-swallow-8b-instruct-v0.5-Q4_K_M.gguf -cnv -c 4096
Note: the model's native context length is 131072, but a smaller context size (e.g. 4096) is recommended for typical local use to avoid excessive memory usage.
License
Please refer to the original model's license terms (Llama 3.3 license and Gemma Terms of Use).
Run FalconSuzuki/Llama-3.1-Swallow-8B-Instruct-v0.5-Q4_K_M-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models