---

language:

• en

license: other

base_model: peikoff/llama3-2-8x3b-moe-objectivist-merged

base_model_relation: quantized

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• mixture-of-experts
• objectivism

---

Objectivist Llama 3.2 8x3B MoE (GGUF)

GGUF quantizations of peikoff/llama3-2-8x3b-moe-objectivist-merged, a QLoRA fine-tune of an 8x3B mixture-of-experts Llama 3.2 merge on Objectivism-related texts. See the merged repo for training details, data, and limitations.

Files

| File | Quant | Size | Status |

|---|---|---|---|

| objectivist_llama3_2_8x3b_moe-Q8_0.gguf | Q8_0 | 19.57 GB | not run |

| objectivist_llama3_2_8x3b_moe-Q6_K.gguf | Q6_K | 15.15 GB | not run |

| objectivist_llama3_2_8x3b_moe-Q5_K_M.gguf | Q5_K_M | 13.16 GB | not run |

| objectivist_llama3_2_8x3b_moe-Q4_K_M.gguf | Q4_K_M | 11.31 GB | loaded and tested |

All files were produced the same way: the merged fp16 model was converted to GGUF with llama.cpp, then quantized with llama-quantize.

Usage

from huggingface_hub import hf_hub_download
path = hf_hub_download("peikoff/objectivist_llama3_2_8x3b_moe-GGUF", "objectivist_llama3_2_8x3b_moe-Q4_K_M.gguf")
print(path)

llama-cli -m <path> -ngl 99 -p "your prompt"

The model uses the Llama 3 chat template (embedded in the GGUF).

Testing

Only the smallest quant (Q4_K_M) was loaded and run; it produced output for the test prompts (domain questions plus one general prompt). The other quants were not run. No benchmark or held-out evaluation has been run, and quantization lowers quality compared with the fp16 merge.

Limitations

• The base model is described by its author as uncensored and abliterated, so refusal behavior is weakened. Add your own safeguards before public deployment.
• Can produce fluent but wrong statements, including invented quotes and biographical details. Verify against primary sources.
• Trained on a single-tradition corpus and reflects that tradition's framing.

License

Set to other. The base model's license, the Llama 3.2 community license, and the terms of the training data may apply. See the merged repo.

Trained by Ason Peikoff.