cstr/Llama3-DiscoLeo-Instruct-8B-v0.1-GGUF overview

---

license: llama3

language:

• de
• en

---

These are quick GGUF quantizations of DiscoResearch/Llama3-DiscoLeo-Instruct-8B-v0.1.

They were done for testing purposes and include:

• one with an older llama.cpp version without bpe pre-tokenizer fix, done per fp16 binary
• one with an older llama.cpp version without bpe pre-tokenizer fix, done per fp32 binary
• one with a recent version and the bpefix

Currently the GGUFs perform below expectations, the -mlx performs best in comparison. Any ideas why?

Provenance and EU AI Act Art. 53 note

• Upstream model: DiscoResearch/Llama3-DiscoLeo-Instruct-8B-v0.1 — published by DiscoResearch.
• Upstream licence: llama3. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.