Models for 4 GB VRAM

Compact GGUF quantizations suitable for 4 GB VRAM laptops and mini PCs. Sorted by Hugging Face downloads. Smallest GGUF file size shown per model.

unsloth/Qwen3.8-27B-GGUF

unsloth · from 13.0 MB · 5,126,652 downloads

prism-ml/Ternary-Bonsai-2-27B-gguf

prism-ml · from 600.1 MB · 3,109,078 downloads

ggml-org/Qwen3.8-27B-GGUF

ggml-org · from 600.1 MB · 2,253,564 downloads

huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF

huihui-ai · from 888.0 MB · 2,106,914 downloads

antirez/deepseek-v4-gguf

antirez · from 889.6 MB · 2,082,306 downloads

empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF

empero-ai · from 875.6 MB · 2,006,265 downloads

handy-computer/nemotron-3.5-asr-streaming-0.6b-gguf

handy-computer · from 472.9 MB · 1,944,876 downloads

handy-computer/parakeet-unified-en-0.6b-gguf

handy-computer · from 455.2 MB · 1,889,751 downloads

DavidAU/Qwen3.5-9B-The-Defiant-Fable-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP-GGUF

DavidAU · from 875.6 MB · 1,887,461 downloads

DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF

DavidAU · from 884.6 MB · 1,799,400 downloads

0bserverx/Qwen3.8-27B-Heretic-Abliterated-Uncensored-GGUF

0bserverx · from 600.1 MB · 1,777,126 downloads

JonathanColetti/Qwen3.8-27B-Uncensored-GGUF

JonathanColetti · from 884.6 MB · 1,666,948 downloads

Serveurperso/YuE2-GGUF

Serveurperso · from 44.4 MB · 1,650,486 downloads

unsloth/gemma-4-26B-A4B-it-GGUF

unsloth · from 440.4 MB · 1,513,752 downloads

antirez/deepseek-v4.1-flash-gguf

antirez · from 925.6 MB · 1,512,607 downloads

peculiar-ragdoll/Dirk-Qwen3.8-27B-GGUF

peculiar-ragdoll · from 884.6 MB · 1,382,089 downloads

unsloth/Qwen3.8-Flash-Next-GGUF

unsloth · from 10.4 MB · 1,370,795 downloads

prism-ml/Bonsai-27B-gguf

prism-ml · from 600.1 MB · 1,218,815 downloads

LiquidAI/LFM2.5-2.6B-GGUF

LiquidAI · from 1.48 GB · 1,212,320 downloads

handy-computer/cohere-transcribe-03-2026-gguf

handy-computer · from 1.45 GB · 1,109,362 downloads

unsloth/GLM-5.3-Flash-GGUF

unsloth · from 9.0 MB · 1,083,884 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V10-GGUF

LuffyTheFox · from 857.6 MB · 1,071,426 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V9-GGUF

LuffyTheFox · from 857.6 MB · 1,038,728 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V12-GGUF

LuffyTheFox · from 857.6 MB · 965,464 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V13-GGUF

LuffyTheFox · from 857.6 MB · 923,388 downloads

ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF

ISTA-DASLab · from 865.5 MB · 891,997 downloads

orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF

orcarouter · from 513.7 MB · 833,465 downloads

lmstudio-community/gemma-4-E4B-it-GGUF

lmstudio-community · from 945.6 MB · 750,134 downloads

AtomicChat/Qwen3.8-Flash-Next-GGUF

AtomicChat · from 553.2 MB · 745,553 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V7-GGUF

LuffyTheFox · from 857.6 MB · 740,390 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V8-GGUF

LuffyTheFox · from 857.6 MB · 740,390 downloads

esatapedico/Qwen3.8-27B-NVFP4-MTP-GGUF

esatapedico · from 888.0 MB · 728,544 downloads

Accio-Lab/occamy-1.0-GGUF

Accio-Lab · from 857.6 MB · 706,808 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-GGUF

LuffyTheFox · from 857.6 MB · 666,062 downloads

unsloth/gemma-4-12b-it-GGUF

unsloth · from 167.0 MB · 654,915 downloads

handy-computer/parakeet-tdt-0.6b-v3-gguf

handy-computer · from 462.9 MB · 653,134 downloads

handy-computer/whisper-medium-gguf

handy-computer · from 480.7 MB · 626,483 downloads

ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF

ISTA-DASLab · from 865.5 MB · 612,673 downloads

prism-ml/Ternary-Bonsai-27B-gguf

prism-ml · from 600.1 MB · 607,071 downloads

LiquidAI/LFM2.5-230M-GGUF

LiquidAI · from 142.2 MB · 582,532 downloads

unsloth/gemma-4-26B-A4B-it-qat-GGUF

unsloth · from 240.3 MB · 557,133 downloads

lmstudio-community/gemma-4-12B-it-QAT-GGUF

lmstudio-community · from 167.0 MB · 522,297 downloads

unsloth/gemma-4-E4B-it-GGUF

unsloth · from 94.1 MB · 518,226 downloads

abenzerps/Apodex-1.1-mini-GGUF

abenzerps · from 857.6 MB · 511,562 downloads

unsloth/gemma-4-31B-it-GGUF

unsloth · from 490.8 MB · 502,588 downloads

Serveurperso/MiniMax-Music3-GGUF

Serveurperso · from 96.0 MB · 496,577 downloads

BoldingBuilds/orcarouter_GLM-5.3-Flash-Uncensored-GGUF

BoldingBuilds · from 1.05 GB · 480,456 downloads

DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking-NEO-CODE-Di-IMatrix-MAX-GGUF

DavidAU · from 884.6 MB · 465,139 downloads

bartowski/Qwen3.8-27B-GGUF

bartowski · from 13.0 MB · 461,454 downloads

AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF

AnkitAI · from 2.33 GB · 460,876 downloads

unsloth/gemma-4-E2B-it-GGUF

unsloth · from 93.3 MB · 454,680 downloads

handy-computer/whisper-large-v3-turbo-gguf

handy-computer · from 511.2 MB · 443,956 downloads

DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF

DavidAU · from 884.6 MB · 437,032 downloads

handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf

handy-computer · from 2.64 GB · 426,237 downloads

meta-models/Muse-Glimmer-30B-GGUF

meta-models · from 1.30 GB · 395,175 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final-GGUF

LuffyTheFox · from 857.6 MB · 392,531 downloads

DavidAU/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF

DavidAU · from 884.6 MB · 387,433 downloads

handy-computer/whisper-large-v3-gguf

handy-computer · from 951.1 MB · 379,530 downloads

ornith-ai/Ornith-1.5-35B-A3B-GGUF

ornith-ai · from 861.0 MB · 369,478 downloads

ornith-ai/Ornith-1.5-9B-GGUF

ornith-ai · from 879.0 MB · 359,078 downloads

AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF

AnkitAI · from 1.96 GB · 350,851 downloads

DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-MTP-GGUF

DavidAU · from 884.6 MB · 342,718 downloads

abenzerps/Spark-X2.5-4B-GGUF

abenzerps · from 2.24 GB · 339,419 downloads

ggml-org/gemma-4-26B-A4B-it-GGUF

ggml-org · from 240.3 MB · 337,534 downloads

huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-GGUF

huihui-ai · from 10.4 MB · 330,731 downloads

bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF

bottlecapai · from 888.0 MB · 330,462 downloads

peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF

peculiar-ragdoll · from 183.3 MB · 325,643 downloads

Jackrong/Qwopus3.6-27B-Coder-Compat-MTP-GGUF

Jackrong · from 888.0 MB · 322,097 downloads

agentionai/Qwen3.8-27B-AP-GGUF

agentionai · from 13.0 MB · 321,725 downloads

LiquidAI/LFM2.5-1.2B-Instruct-GGUF

LiquidAI · from 663.5 MB · 319,638 downloads

peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP

peculiar-ragdoll · from 861.0 MB · 305,167 downloads

handy-computer/canary-180m-flash-gguf

handy-computer · from 132.8 MB · 304,789 downloads

Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF

Jackrong · from 888.0 MB · 304,114 downloads

tencent/Hy-MT2-1.8B-GGUF

tencent · from 1.06 GB · 303,155 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V6-GGUF

LuffyTheFox · from 857.6 MB · 287,745 downloads

unsloth/gemma-4-31B-it-qat-GGUF

unsloth · from 267.0 MB · 287,625 downloads

ukisai/Swift-Qwen3.8-27B-GGUF

ukisai · from 884.6 MB · 271,757 downloads

XHToken/Spark-X2.5-4B-GGUF

XHToken · from 2.42 GB · 258,864 downloads

lmstudio-community/gemma-4-26B-A4B-it-QAT-GGUF

lmstudio-community · from 1.11 GB · 239,170 downloads

Serveurperso/ACE-Step-1.5-GGUF

Serveurperso · from 321.8 MB · 235,422 downloads

LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF

LuffyTheFox · from 857.6 MB · 232,242 downloads

ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF

ukisai · from 553.2 MB · 222,601 downloads

lmstudio-community/gemma-4-31B-it-QAT-GGUF

lmstudio-community · from 1.12 GB · 220,551 downloads

unsloth/Kimi-K3-GGUF

unsloth · from 6.6 MB · 218,732 downloads

handy-computer/Qwen3-ASR-1.7B-gguf

handy-computer · from 1.23 GB · 217,568 downloads

ggml-org/gpt-oss-20b-GGUF

ggml-org · from 878.8 MB · 210,786 downloads

Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-GGUF

Jackrong · from 888.0 MB · 207,166 downloads

Blackfrost-AI/Qwen3.8-27B-ABLITERATED-GGUF

Blackfrost-AI · from 600.1 MB · 203,299 downloads

Jackrong/Qwen3.5-9B-DeepSeek-V4-Flash-GGUF

Jackrong · from 879.0 MB · 201,613 downloads

lmstudio-community/gemma-4-12B-it-GGUF

lmstudio-community · from 167.0 MB · 198,850 downloads

Jackrong/Qwopus3.8-27B-Flash-GGUF

Jackrong · from 1.72 GB · 192,107 downloads

ggml-org/gpt-oss-120b-GGUF

ggml-org · from 809.8 MB · 187,433 downloads

ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF

ggml-org · from 1.08 GB · 185,182 downloads

mudler/gemma-4-26B-A4B-it-APEX-GGUF

mudler · from 1.11 GB · 184,373 downloads

mudler/Qwen3.6-35B-A3B-APEX-GGUF

mudler · from 861.0 MB · 182,394 downloads

openbmb/MiniCPM-V-4.7-35B-A3B-gguf

openbmb · from 752.9 MB · 170,728 downloads

nerkyor/Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-GGUF-NInfer

nerkyor · from 26.0 MB · 166,152 downloads

Jackrong/Qwopus3.6-35B-A3B-Coder-MTP-GGUF

Jackrong · from 1.66 GB · 159,871 downloads

DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF

DavidAU · from 884.6 MB · 159,470 downloads

handy-computer/Qwen3-ASR-0.6B-gguf

handy-computer · from 562.2 MB · 155,839 downloads

Run models locally with guIDE

Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.

Download guIDE → · Browse 524k+ models · Compare models