---

license: other

license_name: modified-mit-mistral-ai-2025

license_link: LICENSE

pipeline_tag: text-generation

base_model: simpledirect/Vinci-MLE-123B-1.0

base_model_relation: quantized

tags:

- gguf

- machine-learning-engineering

- tool-use

- devstral

- research

- canadian-ai

- simpledirect

- vinci

---

<img src="./assets/vinci-vero-header.svg" alt="Vinci maker’s mark and wordmark with Vero" width="360" />

Vinci-MLE-123B-1.0-GGUF

!Vinci MLE 123B — Vero at an ML research workbench. Open weights, Devstral 2, research.

Vinci MLE 123B in GGUF — ML-engineering specialization on Devstral 2, available in three quantizations.

Canadian-developed by SimpleDirect · Mistral AI Modified MIT License · Research release

This repository holds GGUF distributions of Vinci MLE 123B 1.0 in three quantizations, Q4_K_M, Q5_K_M and Q8_0, each split into shards of at most 45 GB. See Files, Quick start and Runtime validation.

Choose your format

| Distribution | Weight download | Documented runtime configuration |

|---|---:|---|

| BF16 Safetensors | 250.05 GB | vLLM; two 141 GB GPUs |

| GGUF Q4_K_M | 74.9 GB | llama.cpp; one NVIDIA H200 |

| GGUF Q5_K_M | 88.3 GB | llama.cpp; one NVIDIA H200 |

| GGUF Q8_0 | 132.9 GB | llama.cpp; two NVIDIA H200s |

Choose BF16 for the Transformers-format weights used in the reported model evaluations, or GGUF for the documented llama.cpp workflow and smaller downloads. These are tested configurations, not minimum-memory requirements. Allow additional memory for context and runtime overhead. BF16 benchmark results do not establish GGUF capability retention.

ML engineering specialization

Vinci MLE 123B is a fine-tune of Mistral AI’s Devstral 2 123B Instruct, trained around evidence-based diagnosis, bounded workspace changes, and justified no-change or stop decisions. The source model uses 91 trajectories across 64 task roots in eight ML-engineering families: data quality; experiment design and interpretation; failed resume; memory and batching; metric and evaluator defects; NaNs and divergence; packaging and serving; and tokenizer and model configuration.

| Trained response | Intended behavior |

|---|---|

| Act | Make a bounded change when the evidence supports an intervention |

| No Change | Preserve a correct setup and explain the evidence |

| Stop / Needs Evidence | Identify missing evidence and the next check |

These describe training scope, not measured success rates. An application supplies the tools, workspace, permissions, and execution loop; the weights alone are not a complete agent.

Files

| Quantization | Shards | Download size | Files |

|---|---:|---:|---|

| Q4_K_M | 2 | 74.9 GB (74,897,134,048 bytes) | Shard folder |

| Q5_K_M | 2 | 88.3 GB (88,316,809,696 bytes) | Shard folder |

| Q8_0 | 3 | 132.9 GB (132,854,420,064 bytes) | Shard folder |

Sizes are in decimal GB (10^9 bytes). Q4_K_M offers the smallest download; Q5_K_M and Q8_0 provide larger representations. Choose according to available memory and validate on your workload. Download size does not equal runtime memory use.

Every distributed file's SHA-256 is listed in SHA256SUMS. GGUF-RUNTIME.json records the conversion toolchain and per-file sizes and digests. The record also identifies intermediate artifacts that are not included in the download.

Quick start

Use Python 3.9 or later, the Hugging Face hf CLI, and a CUDA build of llama.cpp at the tested revision. Authenticate with hf auth login if repository access requires it. The example selects Q4_K_M, the smallest download; allow memory for the weights, context, and runtime overhead.

Download Q4_K_M

Download both shards and the checksum manifest from the same repository revision:

hf download simpledirect/Vinci-MLE-123B-1.0-GGUF \
  --revision 108c5f873aa24bd7c1e8abf2d04ca7d7b6c34968 \
  --include "Vinci-MLE-123B-1.0-Q4_K_M/*" \
  --include "SHA256SUMS" \
  --local-dir ./vinci-mle-123b-gguf
cd ./vinci-mle-123b-gguf

Verify both shards

Run the check below before loading. It requires exactly one checksum entry for each expected shard, requires both files to exist, and verifies their SHA-256 values. It reads files in chunks and works on Linux and macOS. Verification reads the full 74.9 GB download and may take several minutes.

<details>

<summary>Copy and run the complete-shard verification command</summary>

python3 - <<'PYVERIFY'
import hashlib
import re
from pathlib import Path

folder = "Vinci-MLE-123B-1.0-Q4_K_M"
expected = {
    f"{folder}/{folder}-{part:05d}-of-00002.gguf"
    for part in range(1, 3)
}
checksums = {}
for line in Path("SHA256SUMS").read_text().splitlines():
    fields = line.split(maxsplit=1)
    if len(fields) != 2:
        continue
    digest, name = fields
    name = name.removeprefix("*").removeprefix("./")
    if name in expected:
        if name in checksums or not re.fullmatch(r"[0-9a-fA-F]{64}", digest):
            raise SystemExit(f"Invalid checksum entry for {name}; re-download the pinned manifest.")
        checksums[name] = digest.lower()
if set(checksums) != expected:
    raise SystemExit("Required shard checksums are missing; re-download the pinned manifest.")
for name in sorted(expected):
    path = Path(name)
    if not path.is_file():
        raise SystemExit(f"Missing shard {name}; download every shard of this variant.")
    actual = hashlib.sha256()
    with path.open("rb") as model_file:
        for chunk in iter(lambda: model_file.read(8 * 1024 * 1024), b""):
            actual.update(chunk)
    if actual.hexdigest() != checksums[name]:
        raise SystemExit(f"Checksum mismatch for {name}; re-download this shard.")
    print(f"Verified SHA-256: {name}")
print("Verified both Q4_K_M shards; the selected variant is complete.")
PYVERIFY

</details>

Start the local server

After verification succeeds, load the first shard; llama.cpp locates the remaining shard automatically:

llama-server \
  -m Vinci-MLE-123B-1.0-Q4_K_M/Vinci-MLE-123B-1.0-Q4_K_M-00001-of-00002.gguf \
  -ngl 99 --no-mmap \
  --alias vinci-mle-123b-gguf --host 127.0.0.1 --port 8080

This example requests full GPU offload, as in the recorded checks. The Q4_K_M check used one NVIDIA H200; that is a tested configuration, not a minimum-memory requirement. The server command is a starting example, not a reproduction of the fixed-prompt evaluation. The embedded chat template equals the BF16 release's chat_template.jinja.

Send a first request

Once the server is ready, run this in a second terminal:

curl --fail-with-body http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "vinci-mle-123b-gguf",
  "messages": [
    {
      "role": "user",
      "content": "A dataset contains multiple records per customer. Training and validation rows were split at random. What should I inspect before trusting the validation score? Separate observations from hypotheses and do not claim to have run checks."
    }
  ],
  "temperature": 0,
  "max_tokens": 512
}'

This is an illustrative text-only prompt, not a captured model result or benchmark. It does not execute tools or reproduce the training harness. Review the response before using it in an experiment.

With "max_tokens": 512, the recorded run of this exact request stopped at the token limit (finish_reason: "length"), so the answer it returns is cut off. Raise max_tokens if you want a complete answer.

Runtime validation

All three quantizations completed the recorded load-and-generate checks. These checks establish basic runtime operation. Task-level capability retention was not evaluated.

| Quantization | Tested hardware | Generation checks | Tool-format result |

|---|---|---|---|

| Q4_K_M | 1 × NVIDIA H200 | 6 of 6 prompts produced output; all layers offloaded; no errors in the load log | No parseable call |

| Q5_K_M | 1 × NVIDIA H200 | 6 of 6 prompts produced output; all layers offloaded; no errors in the load log | No parseable call |

| Q8_0 | 2 × NVIDIA H200 | 6 of 6 prompts produced output; all layers offloaded; no errors in the load log | No parseable call |

The single tool-call prompt did not produce a parseable call on any of the three quantizations. Tool execution was not validated.

The table summarizes the separate runtime checks reported for this release. GGUF-RUNTIME.json covers conversion and artifact integrity; it does not contain the generation or tool-format smoke logs.

Runtime validation is limited to the llama.cpp revision, interfaces, hardware, and settings reported here. Other configurations were not tested.

<details>

<summary>Runtime-check settings and interface comparison</summary>

llama.cpp source revision 6f4f53f2b7da54fcdbbecaaa734337c337ad6176; llama-completion for Q4_K_M and llama-server for Q5_K_M and Q8_0 (Q4_K_M was also run through llama-server, with the same first 8 generated tokens on all 7 prompts); all layers offloaded to the GPU, with memory-mapping off; greedy decoding (temperature 0, top-k 1); up to 48 generated tokens for each of 6 fixed prompts, plus one fixed tool-call prompt; prompts rendered with the model's own chat template.

</details>

Evaluation and deployment scope

The BF16 source card reports paired tool-calling and reasoning measurements against its Devstral parent, with their methodology and limitations. A valid held-out MLE-task score and a LiveCodeBench coding result are not available for 123B 1.0.

BF16 benchmark results are not GGUF measurements. Task-level benchmark parity, quality differences between quantizations, and minimum memory requirements have not been established. Runtime validation is limited to the configurations documented under Runtime validation.

For the available BF16 distribution and its serving example, see the source model. For the smaller GGUF siblings, see 8B and 30B.

Provenance

Converted with llama.cpp pinned at source revision 6f4f53f2b7da54fcdbbecaaa734337c337ad6176 from the released BF16 files (all 35 re-hashed against their published pins first). The BF16 source snapshot identifies the matching model files. A BF16 GGUF master (sha256 f4c613021ecc1f9b9373c5c390bf82d543c257d98461f1ed299fc30863e5a553) was quantized to each variant, which was then split with llama-gguf-split at 45G. The master and the unsplit quantizations listed in GGUF-RUNTIME.json are not distributed, and no F16 GGUF is distributed.

Licence and attribution

The source model carries Mistral AI’s Modified MIT License, including its revenue condition, rather than the Apache-2.0 terms used by the Granite-based 8B and 30B siblings. Review the source model’s licence section, and this repository’s full LICENSE and NOTICE.

Mistral AI developed Devstral 2; SimpleDirect developed the Vinci MLE fine-tune. No affiliation with or endorsement by Mistral AI is implied.

BF16 source · Vinci MLE collection · SimpleDirect