jersobh/jeff-2b-gguf overview
Jeff 2B Decision Model Optimized Dual Precision Benchmarks Fine tuned on top of internlm/Intern Decision 2B using Unsloth with extended 5 epoch training and ha…
Runs locally from ~640.3 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
---
language:
- en
license: apache-2.0
tags:
- unsloth
- qlora
- gguf
- decision-engine
- routing
base_model: internlm/Intern-Decision-2B
---
Jeff-2B Decision Model (Optimized Dual Precision Benchmarks)
Fine-tuned on top of internlm/Intern-Decision-2B using Unsloth with extended 5-epoch training and hard-negative mining. Optimized for structured schema-guided routing and intent classification.
Benchmark Results Comparison
| Benchmark / Task | Evaluation Slice | Unquantized (BF16) | Quantized (Q4_K_M GGUF) |
| :--- | :--- | :--- | :--- |
| BANKING77 Intent Classification | test[:1000] | 95.80% | 95.60% |
| MASSIVE Intent Classification | test[:1000] | 87.10% | 87.00% |
| CLINC150 Intent Classification | test[:1000] | 95.20% | 95.00% |
| Type-Decision Routing (Dolly) | train[:1000] (Held-out) | 90.30% | 90.10% |
Training Configuration
- Base Model:
internlm/Intern-Decision-2B - Precision: Native BF16 (NVIDIA L4 24GB)
- Method: LoRA (Rank:
32, Alpha:32) - Epochs:
5(Hard-negative mining enabled) - Export Precision: 4-bit Quantized GGUF (
Q4_K_M)
Usage with llama-cli
./llama-cli -m jeff-2B.Q4_K_M.gguf -ngl 99 -n 4 -p "Task: routing\nState: Query user account balance\nOptions: [A: Execute tool, B: Chat directly]\n### Decision:\n"Run jersobh/jeff-2b-gguf with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models