---

license: apache-2.0

base_model: fdtn-ai/antares-1b

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- security

- vulnerability-localization

- granite

---

antares-1b-bf16-GGUF

Unquantized bfloat16 GGUF build of fdtn-ai/antares-1b,

Cisco Foundation AI's file-level vulnerability-localization model.

Every published GGUF of this model so far is quantized (Q2_K … Q8_0). This one is not: weights are

the original bf16 values, converted only in container format.

| File | Size | Weights |

| --- | --- | --- |

| antares-1b-bf16.gguf | 3.42 GiB | BF16 (F32 norms), unquantized |

How it was built

python convert_hf_to_gguf.py fdtn-ai/antares-1b --outtype bf16 --outfile antares-1b-bf16.gguf

llama.cpp build 10090 (720d7fa), converter class GraniteHybridModel.

One caveat if you reproduce this: the upstream config.json (exported with Transformers 5.x) keeps

the RoPE base only inside the nested rope_parameters block. Add a top-level

"rope_theta": 10000000 before converting and then verify it landed in the GGUF —

a silent fall back to the default base would quietly wreck long-context attention:

python -c "from gguf import GGUFReader; r=GGUFReader('antares-1b-bf16.gguf'); print(r.fields['granite.rope.freq_base'].contents())"
# 10000000.0

This build's metadata was checked: rope.freq_base 10000000, attention.scale 0.0078125,

embedding_scale 12.0, residual_scale 0.22, logit_scale 8.0, block_count 40,

head_count 16, head_count_kv 4, vocab_size 100352.

Measurements

Perplexity over 9 chunks of Python source at n_ctx 2048, and llama-bench on an Apple M5 Pro

(Metal), same machine, same llama.cpp build:

| Build | PPL (lower is better) | Generation | Prompt | Size |

| --- | --- | --- | --- | --- |

| BF16 (this repo) | 4.7953 ± 0.1847 | 75.5 t/s | 5420 t/s | 3.42 GiB |

| Q8_0 | 4.8648 ± 0.1894 | 124.8 t/s | 5910 t/s | 1.82 GiB |

So the honest trade-off: BF16 gives you the exact checkpoint and ~1.4 % lower perplexity, Q8_0

gives you 1.65× faster generation at half the memory. If you are running batched agentic workloads

where throughput dominates, Q8_0 is the pragmatic choice; if you are validating results or care

about matching the reference implementation, use this one.

Usage

llama-server -m antares-1b-bf16.gguf --alias antares-1b --host 127.0.0.1 --port 8003 \
  -np 8 -c 65536 --metrics

Note that -c is the total context split across -np slots — the command above gives each of

8 slots 8192 tokens.

With Antares CLI

Antares needs a streaming OpenAI-compatible POST /v1/completions endpoint and builds the Granite

prompt itself, so it needs the raw completion text including <tool_call>…</tool_call> blocks.

llama.cpp does not touch those on the completions route, so no workaround is needed here (unlike

mlx_lm.server, which strips them via its json_tools parser).

[profiles.llama]
model = "antares-1b"
backend = "remote"
endpoint = "http://127.0.0.1:8003/v1/completions"
context_window = 32768

[profiles.llama.generation]
max_tokens = 8192
temperature = 0.3
top_p = 1.0
frequency_penalty = 0.3
stop_tokens = ["<|end_of_text|>", "<|start_of_role|>"]
use_completions_api = true

Keep context_window at or below the per-slot context, otherwise long investigations get rejected

by the server.

Related

• MLX build of the same unquantized weights, for Apple Silicon:

DKAvocadoo/antares-1b-mlx-bf16

Model behavior and limits

Unchanged from upstream: the model localizes vulnerabilities at file level — it returns ranked

candidate file paths for a given CWE, without line numbers, snippets, or fixes, and results vary

between identical runs. Treat every result as a lead to verify, not as proof.

License and attribution

Apache 2.0, inherited from the original model. The weights are the work of Cisco Foundation AI;

see fdtn-ai/antares-1b for the authoritative model

card, evaluations, and intended-use guidance. The upstream repository is gated behind an acceptance

click — please read its terms there. This repository only changes the container format.