---

base_model: mistralai/Mistral-7B-v0.1

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- mistral

- macedonian

- llama-cpp

license: mit

language:

- mk

---

JARVIS-Mistral-Phase1b-GGUF

Model ID: Miki-T/JARVIS-Mistral-Phase1b-GGUF

GGUF-format versions of the JARVIS Mistral 7B Phase 1b model for use with llama.cpp and llama-cpp-python. This is the inference-ready deployment format of Miki-T/JARVIS-Mistral-Phase1b.

---

Files

| File | Size | Description |

|------|------|-------------|

| mistral-7b-jarvis-phase1b.Q4_K_M.gguf | 4.1GB | 4-bit quantized — recommended for daily use |

| mistral-7b-jarvis-phase1b.f16.gguf | 14.5GB | Full float16 — master file for requantization |

---

Model Details

This model is the result of merging two LoRA adapters into base Mistral 7B:

• Phase 1a adapter (Miki-T/JARVIS-Mistral-Phase1a) — Macedonian language foundation trained on 500k rows of Macedonian web text
• Phase 1b adapter (Miki-T/JARVIS-Mistral-Phase1b) — Macedonian instruction following trained on 134k instruction-response pairs

The adapters were merged sequentially using merge_and_unload(), converted to GGUF via convert_hf_to_gguf.py, and quantized to Q4_K_M via llama-quantize.

| Property | Value |

|----------|-------|

| Base model | mistralai/Mistral-7B-v0.1 |

| Language | Macedonian (mk), English (en) |

| Format | GGUF |

| Quantization | Q4_K_M (recommended), f16 (master) |

| Context length | 32768 tokens |

---

Usage

With llama-cpp-python

from llama_cpp import Llama

model = Llama(
    model_path="mistral-7b-jarvis-phase1b.Q4_K_M.gguf",
    n_ctx=2048,
    n_gpu_layers=40,
    verbose=False,
)

prompt = "[INST] Кој е главен град на Македонија? [/INST]"
response = model(prompt, max_tokens=256, temperature=0.7)
print(response["choices"][0]["text"])
# Output: Скопје е главниот град на Македонија.

Prompt format

Uses Mistral instruct format:

[INST] <<SYS>>

Your system prompt here

<</SYS>>

User message [/INST] Assistant response

---

Hardware

• Trained on: NVIDIA RTX 5070 (12GB VRAM)
• Q4_K_M inference: ~4.1GB VRAM, runs on any GPU with 6GB+
• f16 inference: ~14.5GB VRAM

---

Merge and Conversion

# Step 1 — Merge adapters
from transformers import AutoModelForCausalLM
from peft import PeftModel
import torch

model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    torch_dtype=torch.float16,
    device_map="cpu",
)
model = PeftModel.from_pretrained(model, "path/to/phase1a/adapter")
model = model.merge_and_unload()
model = PeftModel.from_pretrained(model, "path/to/phase1b/adapter")
model = model.merge_and_unload()
model.save_pretrained("merged_model/")

# Step 2 — Convert to GGUF (requires llama.cpp)
# python convert_hf_to_gguf.py merged_model/ --outfile model.f16.gguf --outtype f16

# Step 3 — Quantize
# llama-quantize.exe model.f16.gguf model.Q4_K_M.gguf Q4_K_M

---

Related Repositories

• Phase 1a adapter: Miki-T/JARVIS-Mistral-Phase1a
• Phase 1b adapter: Miki-T/JARVIS-Mistral-Phase1b
• Project: https://github.com/MikiTrajkovski/JARVIS

---

License

MIT License

---

Author: Miki Trajkovski | GitHub | HuggingFace

Last Updated: July 3, 2026