---

license: other

license_name: llama3.1

license_link: LICENSE

language:

• eu
• en
• es
• fr
• ca
• gl

base_model:

• HiTZ/Latxa-Llama-3.1-70B-Instruct-v2

pipeline_tag: text-generation

tags:

• latxa
• basque
• euskara
• llama-3.1
• gguf

datasets:

• HiTZ/latxa-datasets-v2

library_name: llama.cpp

---

Latxa-Llama-3.1-70B-Instruct-v2 - GGUF

> ⚠️ This is NOT an official release from HiTZ Zentroa. These GGUF files were converted from the official HiTZ/Latxa-Llama-3.1-70B-Instruct-v2 model by a third party. For the original model, training details, and official support, please refer to the official HiTZ model page.

About Latxa

Latxa is a family of large language models (LLMs) developed by HiTZ Zentroa at the University of the Basque Country, specifically for the Basque language (Euskara), while maintaining strong multilingual support in English, Spanish, French, Catalan, and Galician.

Latxa-Llama-3.1-70B-Instruct-v2 is a 70 billion parameter instruction-tuned model based on Meta's Llama 3.1 architecture, trained on the Latxa v2 datasets.

GGUF Conversions

The following GGUF files are provided, converted using llama.cpp:

| File | Quantization | Size | Description |

|------|-------------|------|-------------|

| Latxa-Llama-3.1-70B-Instruct-v2-FP16.gguf | FP16 | 131 GB | Full precision (reference) |

| Latxa-Llama-3.1-70B-Instruct-v2-Q8_0.gguf | Q8_0 | 70 GB | Near-lossless, recommended for high-quality inference |

| Latxa-Llama-3.1-70B-Instruct-v2-Q4_K_M.gguf | Q4_K_M | 40 GB | Very good quality/size trade-off, most practical |

Usage

With llama.cpp

# Build and run
./build/bin/llama-cli \
  -m Latxa-Llama-3.1-70B-Instruct-v2-Q4_K_M.gguf \
  -p "Azaldu euskararen historia labur-labur." \
  -n 512

With Ollama

Create a Modelfile:

FROM ./Latxa-Llama-3.1-70B-Instruct-v2-Q4_K_M.gguf

Then:

ollama create latxa-70b-v2 -f Modelfile
ollama run latxa-70b-v2

Model Details

• Architecture: Llama 3.1 (decoder-only transformer)
• Parameters: 70B
• Layers: 80
• Context length: 131,072 tokens
• Vocabulary size: 128,256
• Attention: Grouped Query Attention (64 heads, 8 KV heads)

Original Model

For the official model, license, training data, evaluation results, and proper attribution, visit:

https://huggingface.co/HiTZ/Latxa-Llama-3.1-70B-Instruct-v2

License

This model inherits the license from the original Llama 3.1 Community License. Please refer to the original model page for full license terms.

Citation

Cite the original Latxa models:

@misc{latxa2024,
  author = {HiTZ Zentroa},
  title = {Latxa: Basque Language Models},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/HiTZ}},
}

---

These GGUF files were converted from HiTZ/Latxa-Llama-3.1-70B-Instruct-v2-FP8 (decompressed to FP16 during conversion) using llama.cpp.