---

license: apache-2.0

base_model: angelgalvisc/agent-a1-alchemist-4bit

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

language:

- en

- es

- zh

tags:

- gguf

- llama.cpp

- agent

- tool-use

- quantized

- 4-bit

---

Agent-A1 (The Alchemist) — GGUF

The same weights as

angelgalvisc/agent-a1-alchemist-4bit,

repacked for llama.cpp so the model runs where MLX does not: Android, iOS,

NVIDIA, plain CPU.

2.92 GB, and 0.67 GB more if you want it to see.

| file | | |

|---|---|---|

| agent-a1-alchemist-4B-Q4_1.gguf | 2.92 GB | text and tool use |

| mmproj-agent-a1-alchemist.gguf | 0.67 GB | images and video, optional |

It is the same model, not a new quantization

The 4-bit codes were unpacked from the MLX artifact and written straight into

Q4_1 blocks. Nothing was rounded a second time:

• 248 projections at Q4_1. Q4_1 reconstructs w = q·d + m with d and m

in fp16, which is the arithmetic the original uses, so a group of 128 becomes

four blocks of 32 sharing one scale and one minimum. Codes, scales and minima

are identical bit for bit — checked tensor by tensor before writing.

Every scale crosses bf16 → fp16 untouched; the smallest in this model is

7.9e-5 and fp16 reaches down to 6.1e-5.

• Vocabulary table at Q8_0. It is stored at 6 bits and GGUF has no affine

6-bit type — the affine types stop at 5 bits and the 6-bit one is symmetric —

so this is the one tensor that was rounded again. It was measured first:

across 194 positions of real context the most likely next token never

changed, and the top five agreed 99.7% of the time. Keeping it exact instead

costs 600 MB.

Asked the same six questions with greedy decoding, this file and the MLX

artifact answer identically word for word on four of six, and reach the same

result on the other two by a different route. Where they diverge, llama.cpp is

the more precise of the two: it reconstructs the weights in fp32 while MLX does

it in bf16.

Run it

llama-cli -m agent-a1-alchemist-4B-Q4_1.gguf -p "How much is 17 x 23?"

Or from Python:

from llama_cpp import Llama
llm = Llama(model_path="agent-a1-alchemist-4B-Q4_1.gguf", n_ctx=4096, n_gpu_layers=-1)
print(llm.create_completion("...", max_tokens=300, temperature=0.0)["choices"][0]["text"])

Thinking is a switch, and it is on by default here

The chat template writes a reasoning block inside <think>...</think> unless it

is told not to. Every figure this model was measured on came from thinking

off. To turn it off, end the prompt with an empty block:

<|im_start|>assistant
<think>

</think>

With thinking on, leave room: a substantive answer runs about 2800 tokens

against 1600 direct.

Images and video

Add the projector and it sees:

llama-mtmd-cli -m agent-a1-alchemist-4B-Q4_1.gguf \
               --mmproj mmproj-agent-a1-alchemist.gguf \
               --image plan.png -p "What is this?"

The image encoder was never compressed — these are the original weights at

fp16, which is why the projector weighs what it does. Handed a scanned

engineering drawing it answers "a wastewater treatment plant", the same as the

MLX build. It reads what a picture is well and misreads the digits printed on

it, so use it to know what you are looking at, not to take figures off it.

One thing to know

No multi-token prediction. The upstream config declares one MTP block, but

neither this model nor InternScience's own release ships its weights — 723

tensors on both sides, none of them nextn. Converting this architecture with

current llama.cpp leaves nextn_predict_layers = 1 and the runtime then asks

for a block nobody has; here it is set to 0. Anyone converting Agents-A1

themselves will meet the same thing.

What was done to the model

Nothing was retrained. 216 of the 248 matrices are rescaled per channel before

rounding, with the compensating division folded into the neighbouring layer, so

the model computes the same thing at the same file size. The other 32 are the

attention outputs, which have no earlier layer to absorb it. No range is

clipped.

On a 60-task bench where the model plans and calls real tools, this scores 48

against the 43 of the previously released 4-bit version.

License

Apache 2.0 for this packaging. The model is InternScience's, under its own

terms. See NOTICE in the MLX repository for the attribution chain.