---

license: apache-2.0

base_model: athrael-soju/t-176-HydraQwen-3.5-0.8B

---

t-176 HydraQwen-3.5-0.8B — iPhone GGUF artifacts (private)

On-device ship set from ML-101 exp 69 (2026-07-04): one 0.8B model serving

chat (plain context) AND late-interaction retrieval (LoRA attached per

context). Parity-gated on llama.cpp commit 2d973636 against the PyTorch fp32

oracle. Consumed by the exp-69 iOS runbook (ios/bootstrap_mac.sh) and the

in-browser demo Space athrael-soju/hydra-retrieval-web (wllama WASM).

| File | What |

|---|---|

| hydra-base-q6_k.gguf (630 MB) | Qwen/Qwen3.5-0.8B @ 2fc06364, text-only, --no-mtp, Q6_K — ship rung: argmax 16/16, Spearman 0.9955, top-10 overlap 0.962/128p at strict functional tier |

| hydra-lora-f16.gguf (87 MB) | t-176 retrieval LoRA @ c26cd716 (r=64, incl. GDN projections), f16 |

| hydra-mmproj-f16.gguf | vision tower (mmproj) for the GENERATION face — retrieved page images are answered over directly through the shared backbone (no OCR); smoke-tested with llama-mtmd-cli on the pinned commit |

| tokenizer.json / tokenizer_config.json | HF tokenizer (client-side BPE for runtimes without one, e.g. the browser demo; 16/16 id-exact vs the reference recipe) |

| ios_bundle/ | corpus-version-independent assets: 320-dim head (head_w.bin/head_b.bin), page thumbnails (thumbs/, 1024px JPEG), demo example queries (suggestions.json) |

| ios_bundle_v3/ | CURRENT corpus: 128 pages @ vt768, bidir-encoded (pages.f32 123 MB), patch-grid map for heatmaps (pages_map.json), self-test refs (queries_ref.json). Ground-truth DocVQA bench 58/73/76 top-1/3/5 |

| ios_bundle_v2/ | vt1792 bidir corpus (283 MB) — same quality (59/71/77), kept as the announced-eval-vt record |

Attention operating point — asymmetric, matches training and the demo

• Queries: CAUSAL. Single unpadded queries never materialize a 4-D float

mask, so the bidir wrapper no-ops; causal+LoRA matches the PyTorch oracle

at per-token cosine 1.0000 (bf16). This is what the public demo does too.

• Pages/documents: BIDIRECTIONAL — encode in PADDED BATCHES. Padding

materializes the mask and the bidir full-attention patch engages (as in

training). A corpus embedded one-image-per-call silently falls back to

causal and retrieval quality collapses (measured: 19/25/28 vs 58/73/76

top-1/3/5 on ground-truth questions). This repo's v1 ios_bundle/ corpus

had that defect and has been REMOVED; use ios_bundle_v3/.

Q4_K_M was measured and FAILS retrieval parity (argmax 13/16, overlap 0.881);

Q5_K_M is borderline (overlap 0.950, min 0.8). Q8_0 (812 MB) also passes if

size allows.

Retrieval-only usage (no app, stock llama.cpp, NON-merged)

The retrieval face is base + adapter at runtime — the LoRA is never merged

(merging would permanently destroy the pristine-generation face; Hydra's

toggle is the design). Per-token embeddings with any stock llama.cpp:

llama-server -m hydra-base-q6_k.gguf --lora hydra-lora-f16.gguf \
  --embeddings --pooling none -c 512

Query recipe (matches the fixtures this repo was parity-gated on):

1. tokenize the raw query text, NO bos, NO special parsing;
2. append the augmentation suffix: 10 x token id 248044 (<|endoftext|>);
3. POST the token ids to /embeddings -> per-token hidden states (T, 1024);
4. project: h @ head_w.T + head_b (fp32 blobs in ios_bundle/) -> (T, 320);
5. L2-normalize per token; score pages with MaxSim (sum over query tokens of

max page-token dot product) against ios_bundle_v3/pages.f32.

Generation with the adapter attached (or with a merged variant) is OFF-DESIGN

and unevaluated — for generation, load the base WITHOUT the LoRA (+ mmproj

for page images).