---

language:

• bn
• en

license: apache-2.0

library_name: gguf

tags:

• bangla
• bengali
• mistral
• gguf
• llama.cpp
• ollama
• quantized

pipeline_tag: text-generation

datasets:

• ihumaunkabir/alpaca-gpt4-bangla
• wikimedia/wikipedia

base_model:

• mistralai/Mistral-7B-Instruct-v0.3

---

mistral-bangla (GGUF, Q4_K_M)

A standalone, ready-to-run quantized Mistral-7B model adapted for Bangla (Bengali). The

LoRA adapters were merged into the base weights and quantized to Q4_K_M GGUF format -- no

base model loading required, just point Ollama / llama.cpp at the file and run.

Trained via a two-stage recipe: continual pre-training (CPT) on Bangla Wikipedia, then

supervised fine-tuning (SFT) on the

ihumaunkabir/alpaca-gpt4-bangla

instruction dataset, using Unsloth.

> Looking for the trainable adapters? The PEFT LoRA adapters (to continue fine-tuning or

> re-quantize) live at

> ihumaunkabir/mistral_bangla_lora.

| detail | value |

| --- | --- |

| File size | ~4.4 GB |

| Quantization | Q4_K_M (recommended balance of quality + size) |

| Base model needed | No -- weights are merged in |

| Inference tools | Ollama, llama.cpp, LM Studio |

---

How to run

Option A -- Ollama (easiest)

# Pull directly from Hugging Face
ollama pull hf.co/ihumaunkabir/mistral_bangla_q4_k_m_gguf:Q4_K_M

# Run interactively
ollama run hf.co/ihumaunkabir/mistral_bangla_q4_k_m_gguf:Q4_K_M

Or build a local Modelfile with the Alpaca template (see "Prompt format" below) and the system

prompt of your choice, then:

ollama create mistral-bangla -f Modelfile
ollama run mistral-bangla

Option B -- llama.cpp

# Clone + build llama.cpp, then:
./llama-cli \
  -m mistral-7b-v0.3.Q4_K_M.gguf \
  -p "নিচে একটি নির্দেশনা দেওয়া আছে..." \
  -n 128 --color

Option C -- LM Studio

Download the .gguf file and load it through the GUI. No conversion needed.

---

Prompt format

This model was trained on the Alpaca template, translated to Bangla. At inference time,

leave the response blank and let the model generate:

নিচে একটি নির্দেশনা দেওয়া আছে, যা একটি কাজের বর্ণনা দেয়। অনুরোধটি যথাযথভাবে সম্পূর্ণ করে একটি উত্তর লিখুন।

### নির্দেশনা:
<instruction here>

### উত্তর:
<response here>

For an Ollama Modelfile, wrap it as:

TEMPLATE """{{ if .Prompt }}নিচে একটি নির্দেশনা দেওয়া আছে, যা একটি কাজের বর্ণনা দেয়। অনুরোধটি যথাযথভাবে সম্পূর্ণ করে একটি উত্তর লিখুন।

### নির্দেশনা:
{{ .Prompt }}

### উত্তর:
{{ end }}{{ .Response }}"""

PARAMETER stop "### নির্দেশনা:"
PARAMETER stop "### উত্তর:"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1

---

Training lineage (condensed)

This GGUF was produced by:

1. Loading unsloth/mistral-7b-v0.3 in 4-bit (QLoRA).
2. CPT on Bangla Wikipedia (wikimedia/wikipedia, config 20231101.bn).
3. SFT on ihumaunkabir/alpaca-gpt4-bangla (49,969 Bangla instruction pairs).
4. Merging the LoRA adapters back into the base weights at 16-bit.
5. Converting to GGUF and quantizing to Q4_K_M with llama.cpp.

LoRA config: rank 128, alpha 32, rsLoRA on, targets include embed_tokens + lm_head.

---

Limitations

1. Smoke-test training volume (max_steps=120) -- quality will be rough until full epochs are run.
2. CPT data is Wikipedia-only -- formal encyclopedia Bangla; no colloquial text.
3. SFT data is machine-translated (Korean -> Bangla).
4. No safety alignment. No RLHF, DPO, or red-teaming. Filter outputs for your use case.
5. Q4_K_M quantization loses some precision vs. the 16-bit merged model. For higher quality,

re-quantize from the LoRA adapters at Q8_0 or f16.

1. Alpaca prompt format required. The model will produce poor output if prompted with a

different template (e.g. ChatML).

---

Citation

If you use this model, please cite both this GGUF repo and the alpaca-gpt4-bangla dataset,

plus the base Mistral model.

This repo (GGUF)

@misc{mistral-bangla-gguf,
  author       = {ihumaunkabir},
  title        = {mistral-bangla: Bangla Mistral-7B in Q4_K_M GGUF format},
  year         = {2026},
  url          = {https://huggingface.co/ihumaunkabir/mistral_bangla_q4_k_m_gguf},
  note         = {Merged LoRA + Q4_K_M quantization of CPT + SFT Mistral-7B}
}

Source LoRA adapters

@misc{mistral-bangla-lora,
  author       = {ihumaunkabir},
  title        = {mistral-bangla: Bangla CPT + SFT LoRA adapters for Mistral-7B-v0.3},
  year         = {2026},
  url          = {https://huggingface.co/ihumaunkabir/mistral_bangla_lora}
}

SFT dataset -- alpaca-gpt4-bangla

@misc{alpaca-gpt4-bangla,
  author       = {ihumaunkabir},
  title        = {alpaca-gpt4-bangla: A Bangla instruction-following dataset},
  year         = {2026},
  url          = {https://huggingface.co/datasets/ihumaunkabir/alpaca-gpt4-bangla},
  note         = {Machine translation (Korean -> Bangla) of FreedomIntelligence/alpaca-gpt4-korean}
}

Base model

@misc{mistral7b,
  author       = {Mistral AI},
  title        = {Mistral-7B-v0.3},
  year         = {2024},
  url          = {https://huggingface.co/mistralai/Mistral-7B-v0.3}
}