Hopcoder-Mini-35B-A3B — GGUF Quantizations

GGPF quantized versions of TaimoorSiddiqui/Hopcoder-Mini-35B-A3B.

Available formats

| Format | Approx. Size | Bits |

|---|---:|---:|

| BF16 | 69.4 GB | 16 |

| Q8_0 | 36.9 GB | 8 |

| UD-Q8_K_XL | 38.2 GB | 8 |

| UD-Q6_K | 29.3 GB | 6 |

| UD-Q6_K_XL | 31.8 GB | 6 |

| UD-Q5_K_S | 24.9 GB | 5 |

| UD-Q5_K_M | 26.5 GB | 5 |

| UD-Q5_K_XL | 26.5 GB | 5 |

| UD-Q4_K_M | 22.1 GB | 4 |

| UD-Q4_K_S | 20.9 GB | 4 |

| UD-Q4_K_XL | 22.3 GB | 4 |

| MXFP4_MOE | 21.7 GB | 4 |

| UD-IQ4_XS | 17.8 GB | 4 |

| UD-IQ4_NL | 18.1 GB | 4 |

| UD-Q3_K_M | 16.7 GB | 3 |

| UD-Q3_K_XL | 16.8 GB | 3 |

| UD-IQ3_S | 15.0 GB | 3 |

| UD-IQ3_XXS | 13.7 GB | 3 |

| UD-Q2_K_XL | 12.3 GB | 2 |

| UD-IQ2_M | 11.6 GB | 2 |

| UD-IQ2_XXS | 11.5 GB | 2 |

The UD- prefix denotes Unsloth Dynamic quantization (importance-matrix guided).

Usage

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="TaimoorSiddiqui/Hopcoder-Mini-35B-A3B-GGUF",
    filename="Hopcoder-Mini-35B-A3B-Q4_K_M.gguf",
)
print(llm("Hello, who are you?"))

Original model

Base model: Qwen/Qwen-AgentWorld-35B-A3B

Fine-tuned model: TaimoorSiddiqui/Hopcoder-Mini-35B-A3B