---

base_model: AlexWortega/SIQ-1-35B

base_model_relation: quantized

tags:

- gguf

- llama.cpp

- imatrix

- moe

- reasoning

- qwen3

pipeline_tag: text-generation

---

Please check out my refined quant of this model!

https://huggingface.co/el4/SIQ-1-35B-OPAL-GGUF

Please used the patched versions- the original had a metadata bug that I had to manually fix also there is no MTP support

SIQ-1-35B - APEX Quantized (GGUF)

This repository contains GGUF quants of AlexWortega/SIQ-1-35B using the APEX quantization method developed by LocalAI.

These quants are specifically optimized to provide superior perplexity and benchmark scores at lower file sizes compared to standard GGUF quantization methods (like Q4_K_M or Q5_K_M).

About APEX Quantization

APEX (Adaptive Precision EXtension) uses a custom layer-wise bit allocation strategy. Instead of applying a uniform bit-depth across the entire model, APEX analyzes the sensitivity of each layer and assigns higher precision to critical layers while aggressively compressing less sensitive ones.

Result: You get the intelligence of a Q6_K model at the file size of a Q4_K model, making it perfect for running large models on consumer hardware with partial GPU offloading.

📊 File Guide & Hardware Recommendations

SIQ-1-35B consists of 40 transformer layers. Choose the file that best fits your VRAM and System RAM constraints.

| Filename | Size | Quant Profile | Best For | GPU Offload (12GB VRAM) | GPU Offload (24GB VRAM) |

| :--- | :--- | :--- | :--- | :--- | :--- |

| SIQ-1-35B-APEX-I-Quality.gguf | 21.3 GB | I-Quality | Max intelligence, agentic coding, complex reasoning. | ~28 Layers | ~36 Layers |

| SIQ-1-35B-APEX-I-Balanced.gguf | 23.6 GB | I-Balanced | Best balance of speed and high-fidelity output. | ~26 Layers | ~34 Layers |

| SIQ-1-35B-APEX-I-Compact.gguf | 16.1 GB | I-Compact | Fast generation, lower system RAM limits. | ~34 Layers | All 40 Layers |

Note: "GPU Offload" estimates assume standard FP16 KV Cache. If using Q8_0 or Q4_0 KV Cache quantization, you can fit slightly more layers on the GPU.

🚀 Usage Instructions

LM Studio

1. Download the .gguf file that matches your hardware.
2. Import the file into LM Studio.
3. In the right-hand settings panel, set GPU Offload to the maximum recommended layers for your VRAM (see table above).
4. Adjust the Context Length as needed (Tested up to 160k with YaRN scaling).

llama.cpp / ik_llama.cpp (CLI)

./llama-server -m SIQ-1-35B-APEX-I-Quality.gguf -ngl 28 -c 16384 --cache-type-k q8_0 --cache-type-v q8_0

(Adjust -ngl based on your VRAM. Use --cache-type to reduce VRAM usage for large context windows).

📈 Performance & Quality

APEX I-Quality significantly outperforms standard quants of similar size. Based on core metrics for the 35B architecture:

• Perplexity: 6.552 (Matches or beats standard Q5_K_M and Q6_K)
• MMLU: 41.4%
• ARC: 57.9%
• TQA: 38.4%

Credits & Acknowledgments

• Base Model: AlexWortega/SIQ-1-35B by Alex Wortega.
• Quantization Method: apex-quant by LocalAI.
• GGUF Ecosystem: llama.cpp by Georgi Gerganov and contributors.

📄 License

This quantization inherits the license of the base model. Please refer to the original model card for specific licensing details.