---

base_model: mistralai/Mistral-7B-v0.1

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- mistral

- macedonian

- llama-cpp

license: mit

language:

- mk

- en

---

Mistral-7B-MK-Instruct-GGUF

Model ID: Miki-T/Mistral-7B-MK-Instruct-GGUF

GGUF-format versions of Mistral-7B-MK-Instruct for use with llama.cpp and

llama-cpp-python. This is the inference-ready deployment format of

Miki-T/Mistral-7B-MK-Instruct,

the final release of a three-phase Macedonian/English fine-tune. No further

training is planned on this lineage.

Files

| File | Size | Description |

|---|---|---|

| mistral-7b-mk-instruct.Q5_K_M.gguf | 4.9GB | 5-bit quantized — recommended for daily use |

| mistral-7b-mk-instruct.f16.gguf | 14.5GB | Full float16 — master file for requantization |

Model Details

This model is the result of merging three LoRA adapters into base Mistral 7B,

in sequence:

• Phase 1a adapter (Miki-T/JARVIS-Mistral-Phase1a)

— Macedonian language foundation, trained on 500k rows of Macedonian web text

• Phase 1b adapter (Miki-T/JARVIS-Mistral-Phase1b)

— Macedonian instruction following, trained on 134k instruction-response pairs

• Phase 1c adapter (Miki-T/Mistral-7B-MK-Instruct)

— reasoning across 7 benchmark formats + bilingual robustness fix (English

contrast pairs eliminated a language-drift bug where the model would

answer English questions in Macedonian), ~100k mixed MK/EN rows

The adapters were merged sequentially using merge_and_unload(), converted

to GGUF via convert_hf_to_gguf.py, and quantized to Q5_K_M via

llama_model_quantize (the llama-cpp-python binding of the same quantizer

llama-quantize uses).

| Property | Value |

|---|---|

| Base model | mistralai/Mistral-7B-v0.1 |

| Language | Macedonian (mk), English (en) |

| Format | GGUF |

| Quantization | Q5_K_M (recommended), f16 (master) |

| Native context length | 32,768 tokens (base Mistral) |

| Trained sequence length | 1,536 tokens |

Usage

With llama-cpp-python

from llama_cpp import Llama

model = Llama(
    model_path="mistral-7b-mk-instruct.Q5_K_M.gguf",
    n_ctx=4096,
    n_gpu_layers=-1,
    verbose=False,
)

prompt = "[INST] Answer the following question in English.\nWhat is the capital of Macedonia? [/INST]"
response = model(prompt, max_tokens=256, temperature=0.5)
print(response["choices"][0]["text"])
# Output: The capital of Macedonia is Skopje.

prompt = "[INST] Одговори на следново прашање на македонски јазик.\nКој е главен град на Македонија? [/INST]"
response = model(prompt, max_tokens=256, temperature=0.5)
print(response["choices"][0]["text"])
# Output: Скопје е главниот град на Македонија.

Prompt format

Uses Mistral instruct format: [INST] <system/instruction context> User message [/INST] Assistant response

The model answers in whichever language the question is asked in — an

explicit language-following behavior trained via English/Macedonian

contrast pairs in Phase 1c (held-out eval: 0% language drift on English

questions, down from 50% after Phase 1b).

Hardware

• Trained on: NVIDIA RTX 5070 (12GB VRAM)
• Q5_K_M inference: ~4.9GB VRAM, runs on any GPU with 6GB+
• f16 inference: ~14.5GB VRAM

Merge and Conversion

# Step 1 — Merge adapters (sequential: 1a, then 1b, then 1c)
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    torch_dtype=torch.float16,
    device_map="cpu",
)
model = PeftModel.from_pretrained(model, "path/to/phase1a/adapter")
model = model.merge_and_unload()
model = PeftModel.from_pretrained(model, "path/to/phase1b/adapter")
model = model.merge_and_unload()
model = PeftModel.from_pretrained(model, "path/to/phase1c/adapter")
model = model.merge_and_unload()
model.save_pretrained("mistral-7b-mk-instruct/")

tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")
tokenizer.save_pretrained("mistral-7b-mk-instruct/")

# Step 2 — Convert to GGUF (requires llama.cpp)
python convert_hf_to_gguf.py mistral-7b-mk-instruct/ --outfile mistral-7b-mk-instruct.f16.gguf --outtype f16

# Step 3 — Quantize
llama-quantize.exe mistral-7b-mk-instruct.f16.gguf mistral-7b-mk-instruct.Q5_K_M.gguf Q5_K_M

Evaluation

Held-out evaluation (300 rows per language, never seen in training) compared

this model against the pre-Phase-1c checkpoint:

| Metric | Before | After |

|---|---|---|

| EN language drift (Cyrillic on English questions) | 50% | 0% |

| EN average match score | 27.7% | 88.0% |

| MK average match score | 35.6% | 64.2% |

Related Repositories

• Phase 1a adapter: Miki-T/JARVIS-Mistral-Phase1a
• Phase 1b adapter: Miki-T/JARVIS-Mistral-Phase1b
• Phase 1b GGUF: Miki-T/JARVIS-Mistral-Phase1b-GGUF
• Final adapter: Miki-T/Mistral-7B-MK-Instruct
• Project: https://github.com/MikiTrajkovski/JARVIS

License

MIT License

Author: Miki Trajkovski | GitHub | HuggingFace

Last Updated: July 7, 2026