---

language:

• bn
• en

license: apache-2.0

library_name: gguf

tags:

• bangla
• bengali
• qwen3
• gguf
• llama.cpp
• ollama
• quantized

pipeline_tag: text-generation

---

qwen3-bangla (GGUF, Q4_K_M)

A standalone, ready-to-run quantized Qwen3-4B-Base model adapted for Bangla (Bengali). The

LoRA adapters were merged into the base weights and quantized to Q4_K_M GGUF format -- no

base model loading required, just point Ollama / llama.cpp at the file and run.

Fine-tuned via SFT on the

ihumaunkabir/alpaca-gpt4-bangla

instruction dataset, using Unsloth.

> Looking for the trainable adapters? The PEFT LoRA adapters (to continue fine-tuning or

> re-quantize) live at

> ihumaunkabir/qwen3_bangla_lora.

| detail | value |

| --- | --- |

| File size | ~2.5 GB |

| Quantization | Q4_K_M (recommended balance of quality + size) |

| Base model needed | No -- weights are merged in |

| Inference tools | Ollama, llama.cpp, LM Studio |

---

How to run

Option A -- Ollama (easiest)

# Pull directly from Hugging Face
ollama pull hf.co/ihumaunkabir/qwen3_bangla_q4_k_m_gguf:Q4_K_M

# Run interactively
ollama run hf.co/ihumaunkabir/qwen3_bangla_q4_k_m_gguf:Q4_K_M

Or build a local Modelfile with the Alpaca template (see "Prompt format" below) and the system

prompt of your choice, then:

ollama create qwen3-bangla -f Modelfile
ollama run qwen3-bangla

Option B -- llama.cpp

# Clone + build llama.cpp, then:
./llama-cli \
  -m qwen3-4b-base.Q4_K_M.gguf \
  -p "নিচে একটি নির্দেশনা দেওয়া আছে..." \
  -n 128 --color

Option C -- LM Studio

Download the .gguf file and load it through the GUI. No conversion needed.

---

Prompt format

This model was trained on the Alpaca template, translated to Bangla. At inference time,

leave the response blank and let the model generate:

নিচে একটি নির্দেশনা দেওয়া আছে, যা একটি কাজের বর্ণনা দেয়। অনুরোধটি যথাযথভাবে সম্পূর্ণ করে একটি উত্তর লিখুন।

### নির্দেশনা:
<instruction here>

### উত্তর:
<response here>

For an Ollama Modelfile, wrap it as:

TEMPLATE """{{ if .Prompt }}নিচে একটি নির্দেশনা দেওয়া আছে, যা একটি কাজের বর্ণনা দেয়। অনুরোধটি যথাযথভাবে সম্পূর্ণ করে একটি উত্তর লিখুন।

### নির্দেশনা:
{{ .Prompt }}

### উত্তর:
{{ end }}{{ .Response }}"""

PARAMETER stop "### নির্দেশনা:"
PARAMETER stop "### উত্তর:"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1

---

Training lineage (condensed)

This GGUF was produced by:

1. Loading Qwen/Qwen3-4B-Base in 4-bit (QLoRA).
2. SFT on ihumaunkabir/alpaca-gpt4-bangla (49,969 Bangla instruction pairs).
3. Merging the LoRA adapters back into the base weights at 16-bit.
4. Converting to GGUF and quantizing to Q4_K_M with llama.cpp.

LoRA config: rank 128, alpha 32, rsLoRA on, targets include embed_tokens + lm_head.

---

Limitations

1. Smoke-test training volume (max_steps=120) -- quality will be rough until full epochs are run.
2. No safety alignment. No RLHF, DPO, or red-teaming. Filter outputs for your use case.
3. Q4_K_M quantization loses some precision vs. the 16-bit merged model. For higher quality,

re-quantize from the LoRA adapters at Q8_0 or f16.

1. Alpaca prompt format required. The model will produce poor output if prompted with a

different template (e.g. ChatML).

---

Citation

If you use this model, please cite both this GGUF repo and the alpaca-gpt4-bangla dataset,

plus the base Qwen3 model.

This repo (GGUF)

@misc{qwen3-bangla-gguf,
  author       = {ihumaunkabir},
  title        = {qwen3-bangla: Bangla Qwen3-4B in Q4_K_M GGUF format},
  year         = {2026},
  url          = {https://huggingface.co/ihumaunkabir/qwen3_bangla_q4_k_m_gguf},
  note         = {Merged LoRA + Q4_K_M quantization of SFT Qwen3-4B-Base}
}

Source LoRA adapters

@misc{qwen3-bangla-lora,
  author       = {ihumaunkabir},
  title        = {qwen3-bangla: Bangla SFT LoRA adapters for Qwen3-4B-Base},
  year         = {2026},
  url          = {https://huggingface.co/ihumaunkabir/qwen3_bangla_lora}
}

SFT dataset -- alpaca-gpt4-bangla

@misc{alpaca-gpt4-bangla,
  author       = {ihumaunkabir},
  title        = {alpaca-gpt4-bangla: A Bangla instruction-following dataset},
  year         = {2026},
  url          = {https://huggingface.co/datasets/ihumaunkabir/alpaca-gpt4-bangla},
  note         = {Machine translation (Korean -> Bangla) of FreedomIntelligence/alpaca-gpt4-korean}
}

Base model

@misc{qwen3,
  author       = {Qwen Team},
  title        = {Qwen3-4B-Base},
  year         = {2025},
  url          = {https://huggingface.co/Qwen/Qwen3-4B-Base}
}