---

license: apache-2.0

base_model: prithivMLmods/Qwen3.5-4B-Unredacted-MAX

tags:

- gguf

- qwen2

- quantization

- text-generation

language: en

pipeline_tag: text-generation

library_name: gguf

---

Qwen3.5-4B-Unredacted-MAX - GGUF

GGUF quantized versions of prithivMLmods/Qwen3.5-4B-Unredacted-MAX, a 4.5 billion parameter language model based on the Qwen3.5-4B architecture, optimized for improved loading stability and compatibility with modern Transformers pipelines.

Model Details

• Base Model: Qwen/Qwen3.5-4B
• Fine-tuned by: prithivMLmods
• Architecture: Qwen2 (28 layers, 28 attention heads)
• Context Length: 32768 tokens
• Vocabulary Size: 151936
• Parameters: 4.5B

Quantization

| Filename | Bits | Size | Use Case |

|----------|------|------|----------|

| model_f16.gguf | 16 | ~8.4 GB | Maximum quality, high VRAM requirement |

| model_q8_0.gguf | 8 | ~4.5 GB | High quality, moderate VRAM |

| model_q6_k.gguf | 6 | ~3.4 GB | Good quality, balanced VRAM |

| model_q5_k_m.gguf | 5 | ~2.8 GB | Recommended for most use cases |

| model_q5_k_s.gguf | 5 | ~2.5 GB | Compact, minimal quality loss |

| model_q4_k_m.gguf | 4 | ~2.1 GB | Good balance, low VRAM |

| model_q4_k_s.gguf | 4 | ~1.9 GB | Compact model |

| model_q3_k_l.gguf | 3 | ~1.7 GB | Very compact |

| model_q3_k_m.gguf | 3 | ~1.5 GB | Minimal size |

| model_q3_k_s.gguf | 3 | ~1.4 GB | Extreme compression |

| model_q2_k.gguf | 2 | ~1.1 GB | Maximum compression |

VRAM Requirements

| Quantization | VRAM | Recommended Device |

|--------------|------|-------------------|

| F16 | 9 GB | High-end GPU |

| Q8_0 | 5 GB | Mid-range GPU |

| Q6_K | 3.5 GB | Mid-range GPU |

| Q5_K_M | 3 GB | Standard GPU |

| Q4_K_M | 2.5 GB | Standard GPU / Laptop GPU |

| Q3_K_M | 2 GB | Laptop GPU / CPU |

| Q2_K | 1.5 GB | CPU with sufficient RAM |

Usage

llama.cpp

./main -m model_q5_k_m.gguf -n 256 -p "You are a helpful assistant."

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="model_q5_k_m.gguf",
    n_ctx=32768,
    n_threads=8
)

response = llm("What is machine learning?", max_tokens=256)
print(response["choices"][0]["text"])

LM Studio

Download the quantized GGUF file and load it in LM Studio's model picker.

Ollama

ollama pull tinyopsec/qwen3.5-4b-unredacted-max-gguf:q5_k_m
ollama run tinyopsec/qwen3.5-4b-unredacted-max-gguf:q5_k_m

Model Capabilities

This model excels at:

• Instruction Following: Optimized for direct response generation and multi-step instructions
• Reasoning Tasks: Capable of complex reasoning and problem-solving
• Text Generation: High-quality content generation across diverse topics
• Research & Experimentation: Designed for transformer behavior analysis and instruction dynamics understanding
• Lightweight Deployment: Efficient 4.5B parameter architecture suitable for local inference

License

This quantized version maintains the original model's Apache 2.0 license. See the original model card for full details.

Disclaimer

These are community quantizations. For official support and documentation, refer to the original model repository and llama.cpp documentation.