---

license: apache-2.0

pipeline_tag: text-generation

base_model: simpledirect/Vinci-MLE-8B-1.0

base_model_relation: quantized

tags:

- gguf

- machine-learning-engineering

- tool-use

- granite

- research

- canadian-ai

- simpledirect

- vinci

---

<img src="./assets/vinci-vero-header.svg" alt="Vinci maker’s mark and wordmark with Vero" width="360" />

Vinci-MLE-8B-1.0-GGUF

!Vinci MLE 8B — machine-learning engineering research.

Open-weight ML-engineering research in GGUF: evidence-based diagnosis, bounded changes, and justified no-change or stop decisions.

Canadian-developed by SimpleDirect · Apache-2.0 · Research baseline

Available now: F16, Q8_0, Q5_K_M, and Q4_K_M. Each variant has recorded generation and tool-call-format smoke checks; see runtime validation for their scope.

This is the GGUF distribution of Vinci MLE 8B 1.0, an open-weight research model trained around evidence-based diagnosis, bounded workspace changes, and justified no-change or stop decisions. Its source model is a fine-tuned derivative of IBM Granite 4.1 8B.

Model at a glance

| Property | Value |

|---|---|

| Developer | SimpleDirect, Canada |

| Source model | simpledirect/Vinci-MLE-8B-1.0 |

| Original parent | ibm-granite/granite-4.1-8b |

| Distribution format | GGUF |

| Available model files | F16, Q8_0, Q5_K_M, Q4_K_M |

| Conversion runtime | llama.cpp, revision 6f4f53f2 |

| Validation scope | CUDA full-offload runtime smoke; no minimum-memory certification |

| License | Apache-2.0, with NOTICE |

ML engineering specialization

The source model was trained on 91 trajectories across 64 task roots in eight ML-engineering families: data duplication and quality; experiment design and interpretation; failed resume; memory, batching and gradient accumulation; metric/evaluator defects; NaNs and divergence; packaging, serving and reproducibility; and tokenizer/model configuration.

| Trained response | Intended behavior |

|---|---|

| Act | Make a bounded change when evidence supports a defect and intervention |

| No Change | Preserve a correct setup and explain the evidence |

| Stop / Needs Evidence | Identify missing evidence or a blocker instead of inventing a repair |

These describe the source model's training scope, not measured success rates. Both source-model sizes consumed the same tokenized examples but used different training recipes. See the source card for the 8B-specific recipe, selective-supervision design, and development-selection limitations.

Files and choosing a variant

| File | Bytes | GB | GiB |

|---|---:|---:|---:|

| Q4_K_M | 5,347,914,336 | 5.35 | 4.98 |

| Q5_K_M | 6,253,884,000 | 6.25 | 5.82 |

| Q8_0 | 9,345,610,336 | 9.35 | 8.70 |

| F16 | 17,587,417,696 | 17.59 | 16.38 |

Sizes were read from the Hugging Face API on September 28, 2026 and matched to GGUF-RUNTIME.json. File size is not total RAM or VRAM use: allow additional memory for context, the KV cache, and runtime overhead.

Q4_K_M is the smallest download in this set; Q5_K_M and Q8_0 offer progressively larger representations. F16 is the floating-point GGUF conversion. Choose according to available memory and your own task validation; no quality ranking between these variants has been established here. The BF16 conversion master mentioned in the runtime record is an intermediate artifact, not an additional downloadable variant.

Quick start

Install huggingface_hub and transformers, and authenticate with hf auth login if repository access requires it. Use a working CUDA build of llama.cpp at the tested revision with llama-completion on your PATH and enough GPU memory for full offload.

The example downloads Q4_K_M at a fixed repository revision, checks its SHA-256, renders one of the smoke prompts using the source tokenizer's chat template, and invokes the same completion interface and decoding settings used in the recorded checks. It downloads only the selected GGUF variant; source tokenizer loading does not download the BF16 weight shards.

import hashlib
import subprocess

from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

model_path = hf_hub_download(
    repo_id="simpledirect/Vinci-MLE-8B-1.0-GGUF",
    filename="Vinci-MLE-8B-1.0-Q4_K_M.gguf",
    revision="4c8abbe3e2c45cae749d7b9a865ca86c0bdc98b4",
    local_dir="./vinci-mle-8b-gguf",
)
with open(model_path, "rb") as model_file:
    actual_sha256 = hashlib.file_digest(model_file, "sha256").hexdigest()
if actual_sha256 != "87313e7e349fa9e59a49cf73ec1e41b5764c1e3b5642ac0543229df460efc798":
    raise RuntimeError("GGUF checksum mismatch; download the pinned file again.")

tokenizer = AutoTokenizer.from_pretrained(
    "simpledirect/Vinci-MLE-8B-1.0",
    revision="75256cf293770b6439b5ca52b98d43aa7687294d",
    trust_remote_code=False,
)
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain in two sentences what a learning-rate warmup does."}],
    tokenize=False,
    add_generation_prompt=True,
)
subprocess.run(
    [
        "llama-completion", "-m", model_path, "-p", prompt,
        "-n", "48", "--temp", "0", "--top-k", "1", "--seed", "42",
        "-no-cnv", "--no-display-prompt", "-ngl", "99", "--no-mmap",
    ],
    check=True,
)

Use Python 3.11 or later for hashlib.file_digest. This is a reference integration of the recorded smoke procedure, not a new run on your hardware. The example requests full GPU offload; CPU-only use, partial offload, other backends, and graphical front ends were not validated by these checks. The source model's Transformers/vLLM weight-loading instructions apply to the BF16 repository, not these GGUF files.

For another variant, change the filename and expected hash together using SHA256SUMS. When verifying a complete snapshot, run sha256sum -c SHA256SUMS (macOS: shasum -a 256 -c SHA256SUMS) inside its directory. The single-file example above verifies only its selected GGUF.

Runtime validation

The uploaded runtime record reports the following for each distributed variant:

| Variant | Nonempty generation outputs | Nonzero generation exit codes | Detected NaN/error log lines | Tool-call format |

|---|---:|---:|---:|---|

| Q4_K_M | 6/6 | 0 | 0 | Parsed read_file call |

| Q5_K_M | 6/6 | 0 | 0 | Parsed read_file call |

| Q8_0 | 6/6 | 0 | 0 | Parsed read_file call |

| F16 | 6/6 | 0 | 0 | Parsed read_file call |

The checks used six fixed prompts with a 48-token generation budget, plus one tool-format prompt with a 96-token budget per variant. Decoding was greedy (temperature 0, top-k 1, seed 42), with CUDA full offload and memory mapping disabled. Prompts were rendered with the staged source tokenizer; embedded chat templates matched the staged template in every distributed file.

These are runtime and format checks, not capability evaluations. The tool check parsed a JSON call and checked the name read_file; it did not validate the arguments, execute the tool, or establish reliable agent behavior. The short generation checks do not measure answer correctness, throughput, minimum memory, or long-context capability.

<details>

<summary>Additional conversion checks</summary>

The first eight re-tokenized output tokens agreed between Q8_0 and F16 on all six generation prompts. This narrow agreement does not establish complete-output equivalence or quality retention against BF16.

An independent 8B conversion produced byte-identical files for the distributed variants and the intermediate BF16 master. This checks conversion reproducibility, not deterministic generation across runtimes.

</details>

Evaluation and limitations

Vinci MLE 1.0 has no valid held-out ML-engineering task score. The source card reports broad parent-capability comparisons separately from MLE training scope, including unfavorable results and methodology limitations.

GGUF runtime smoke checks are reported above. Task-level capability retention against the BF16 source has not been measured for these variants. The BF16 card’s benchmark scores are not GGUF measurements; no benchmark parity or MLE-task improvement is claimed.

The source model's evaluation scope, limitations, and contamination audit remain relevant. Applications must implement and validate their own tools, sandbox, permissions, and workspace; model weights alone are not a complete Vinci MLE agent harness.

Source and conversion provenance

| Record | Value |

|---|---|

| BF16 source artifact reference | 75256cf293770b6439b5ca52b98d43aa7687294d |

| GGUF artifact snapshot | 4c8abbe3e2c45cae749d7b9a865ca86c0bdc98b4 |

| Conversion tool | llama.cpp 6f4f53f2b7da54fcdbbecaaa734337c337ad6176 |

| F16 conversion | Direct conversion from the staged BF16 Safetensors with convert_hf_to_gguf.py --outtype f16 |

| Quantization input | BF16 GGUF master converted from the staged Safetensors |

| Quantization variants | llama-quantize with Q8_0, Q5_K_M, or Q4_K_M; no importance matrix supplied |

| Artifact checksums | SHA256SUMS |

| Runtime checks | GGUF-RUNTIME.json |

| Task-level comparison with BF16 | Not measured |

The conversion workflow checked the staged source files against the release artifact pins before conversion. The BF16 source reference identifies those model artifacts; the GGUF snapshot pins the uploaded conversions and their original runtime record. Later card-only changes do not alter those files. Source-model provenance is available in the BF16 repository’s PROVENANCE.json, also referenced by the accompanying source NOTICE.

License and attribution

The source model is licensed under Apache-2.0; its LICENSE and NOTICE are included here. IBM developed Granite; SimpleDirect developed the Vinci MLE fine-tune. No affiliation with or endorsement by IBM is implied.

BF16 source · MLE 30B GGUF · Vinci models