unokayish182/llama-sahabat-ai-v2-70B-GGUF-Relist overview
ALL QUANTIZATIONS BY https://huggingface.co/blackshell69 Llama Sahabat AI v2 70B IT — GGUF Quantizations GGUF quantizations of Sahabat AI/Llama Sahabat AI v2 7…
Runs locally from ~22.79 GB disk (24 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
| Model ID | unokayish182/llama-sahabat-ai-v2-70B-GGUF-Relist |
|---|---|
| Author | unokayish182 |
| Pipeline | — |
| License | llama3.1 |
| Base model | Sahabat-AI/Llama-Sahabat-AI-v2-70B-IT |
| Last modified | 2026-06-18T00:35:50.000Z |
Model README
---
license: llama3.1
language:
- id
- jv
- su
- en
base_model:
- Sahabat-AI/Llama-Sahabat-AI-v2-70B-IT
---
ALL QUANTIZATIONS BY https://huggingface.co/blackshell69
Llama-Sahabat-AI-v2-70B-IT — GGUF Quantizations
GGUF quantizations of Sahabat-AI/Llama-Sahabat-AI-v2-70B-IT, quantized locally using llama.cpp.
About the base model
Sahabat-AI adalah model bahasa besar (LLM) yang dikembangkan secara kolaboratif oleh BRIN, GoTo, dan Bukalapak untuk mendukung ekosistem AI berbahasa Indonesia. Model ini dilatih menggunakan data bahasa Indonesia yang kaya dan beragam, sehingga mampu memahami konteks budaya, bahasa, dan kebutuhan spesifik pengguna Indonesia dengan lebih baik.
Sahabat AI is a Large Language Model (LLM) collaboratively developed by BRIN (National Research and Innovation Agency), GoTo, and Bukalapak to support the Indonesian-language AI ecosystem. Trained on rich and diverse Indonesian language data, it better understands the cultural context, language nuances, and specific needs of Indonesian users.
Available quantizations
| Repo | Quantization | Size | BPW | imatrix | Best for |
|---|---|---|---|---|---|
| blackshell69/Llama-Sahabat-AI-v2-70B-IT-Q8_0 | Q8_0 | 70 GB | 8.50 | No | Near-lossless, 80+ GB VRAM |
| blackshell69/Llama-Sahabat-AI-v2-70B-IT-Q3_K_S | Q3_K_S | 29 GB | 3.50 | No | Best quality that fits a 32 GB GPU |
| blackshell69/Llama-Sahabat-AI-v2-70B-IT-Q2_K_S | Q2_K_S | 23 GB | 2.77 | Yes | Smallest footprint, quality trade-off |
Quantization notes
- Q8_0 — converted directly from bf16 safetensors; effectively lossless
- Q3_K_S — good quality-size trade-off; fits comfortably on a single 32 GB V100
- Q2_K_S — uses an importance matrix (imatrix) generated from Q3_K_S + groups_merged.txt calibration data for better weight selection at extreme compression
Usage
Load with any llama.cpp-compatible runner:
llama-cli -m llama-sahabat-70b-Q3_K_S.gguf -p "Halo, apa kabar?" -ngl 40Hardware requirements
| Quantization | Min VRAM (full offload) | CPU RAM (no GPU) |
|---|---|---|
| Q2_K_S | ~25 GB | ~28 GB |
| Q3_K_S | ~32 GB | ~34 GB |
| Q8_0 | ~75 GB | ~78 GB |
Run unokayish182/llama-sahabat-ai-v2-70B-GGUF-Relist with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models