piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUF overview
Model highly unstable, use only for basic information Bielik V3.0 11B Instruct, a language model made by SpeakLeash aka Spichlerz , but heavily quantized. The …
Runs locally from ~7.5 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
---
license: apache-2.0
language:
- pl
base_model:
- speakleash/Bielik-PL-11B-v3.0-Instruct
tags:
- heavily_quantized
- IQ2_XXS
---
Model highly unstable, use only for basic information!
Bielik V3.0 11B Instruct, a language model made by SpeakLeash (aka Spichlerz), but heavily quantized. The model's raw weights were taken and compressed to IQ2_XXS, an aggresive form of quantization meant for GGUF models. It achieves ~10tok/s while using 3.1GB of RAM with a context window of 8 thousand on the MacBook Pro (base M1, 8GB of unified memory, 256GB of internal storage).
Bielik at this level of compression tends to hallucinate frequently and generate context in languages different than intended. (Polish and english are the intended languages.)
Run piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models