VertexResearch/Vertex-0.6-100M-8192-Instruct-GGUF overview
Vertex 0.6 100M — 8192 ctx Instruct GGUF GGUF quantizations of Vertex 0.6 100M 8192 Instruct https://huggingface.co/VertexResearch/Vertex 0.6 100M 8192 Instruc…
Runs locally from ~59.4 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| Vertex-0.6-100M-8192-Instruct-BF16.gguf | GGUF | BF16 | 185.7 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q2_K.gguf | GGUF | Q2_K | 59.4 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q3_K_M.gguf | GGUF | Q3_K_M | 63.1 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q3_K_S.gguf | GGUF | Q3_K_S | 59.4 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q4_0.gguf | GGUF | Q4_0 | 62.1 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q4_1.gguf | GGUF | Q4_1 | 66.7 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q4_K_M.gguf | GGUF | Q4_K_M | 71.8 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q4_K_S.gguf | GGUF | Q4_K_S | 69.2 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q5_K_M.gguf | GGUF | Q5_K_M | 76.4 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q5_K_S.gguf | GGUF | Q5_K_S | 74.7 MB | Download |
| Vertex-0.6-100M-8192-Instruct-Q8_0.gguf | GGUF | Q8_0 | 99.2 MB | Download |
Model Details
| Model ID | VertexResearch/Vertex-0.6-100M-8192-Instruct-GGUF |
|---|---|
| Author | VertexResearch |
| Pipeline | text-generation |
| License | apache-2.0 |
| Base model | VertexResearch/Vertex-0.6-100M-8192-Instruct |
| Last modified | 2026-08-26T12:42:14.000Z |
Model README
---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
library_name: llama.cpp
tags:
- gguf
- instruct
- chat
- tool-calling
- qwen3
base_model: VertexResearch/Vertex-0.6-100M-8192-Instruct
---
Vertex 0.6 100M — 8192-ctx Instruct (GGUF)
GGUF quantizations of Vertex-0.6-100M-8192-Instruct for llama.cpp. 97M-parameter Qwen3-architecture chat model by Vertex Research: ChatML format, tool calling, 8192 context (RoPE theta 1M).
Quants
| Quant | Size |
|---|---|
| BF16 | 195 MB |
| Q8_0 | 104 MB |
| Q5_K_M | 80 MB |
| Q5_K_S | 78 MB |
| Q4_K_M | 75 MB |
| Q4_K_S | 73 MB |
| Q4_1 | 70 MB |
| Q4_0 | 65 MB |
| Q3_K_M | 66 MB |
| Q3_K_S | 62 MB |
| Q2_K | 62 MB |
At 97M parameters the embedding table dominates file size, so low-bit quants save less than usual — and quantization hurts small models disproportionately. Q8_0 or higher recommended; BF16 for best quality.
Usage
llama-cli -m Vertex-0.6-100M-8192-Instruct-Q8_0.ggufChat template (ChatML) is embedded. EOS: </s> and <|im_end|>.
Limitations
These models are not the most coherent yet and need more tuning: expect rambling, repetition, and inconsistent answers, especially over longer generations.
Run VertexResearch/Vertex-0.6-100M-8192-Instruct-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models