Solstice-AI/gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF overview

---

language:

• en
• zh

license: apache-2.0

tags:

• anvil
• turboquant
• gguf
• q8_0
• gemma
• claude-distill
• claude-opus-4.6
• thinking-distilled
• reasoning
• stage7
• chain-of-thought
• metal
• cuda

pipeline_tag: text-generation

---

<p align="center">

<img src="https://cdn-uploads.huggingface.co/production/uploads/67c2e844e0921a5410eec10a/Y5M42dCag2f7Fc6fDtV0Z.jpeg" alt="Solstice-AI Banner" width="100%">

</p>

<h1 align="center">gemma-4-31b-claude-opus-4.6-thinking-distilled-s7 (GGUF Q8_0)</h1>

<h3 align="center">Gemma 4 31B Dense Architecture with Stage-7 Claude Opus 4.6 Reasoning Distillation</h3>

<p align="center">

<img src="https://img.shields.io/badge/org-Solstice--AI-blueviolet" alt="Solstice-AI">

<img src="https://img.shields.io/badge/license-Apache%202.0-blue" alt="License">

<a href="https://github.com/Solstice-Labs/anvil"><img src="https://img.shields.io/badge/engine-Anvil%20Runtime%20(TurboQuant)-crimson" alt="Anvil Runtime"></a>

<img src="https://img.shields.io/badge/format-GGUF%20(Q8__0)-orange" alt="Format">

<img src="https://img.shields.io/badge/context-131K%20Tokens-success" alt="Context Window">

<img src="https://img.shields.io/badge/file%20size-32.6%20GB-informational" alt="File Size">

</p>

---

Executive Summary

gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF is a flagship 31.4-billion parameter dense reasoning checkpoint. It combines the architectural stability of Google's Gemma 4 31B with high-entropy, multi-step Chain-of-Thought (CoT) reasoning traces distilled directly from Anthropic's Claude Opus 4.6 at training Stage 7 (s7).

Packaged in precision GGUF Q8_0 (high-fidelity 8-bit quantization) and accelerated by the Anvil runtime, this model reproduces Claude Opus 4.6's distinctive recursive verification, counterfactual self-auditing, and structured step-by-step code synthesis on consumer and enterprise local hardware.

---

Stage-7 Distillation Heritage & Behavioral Mechanics

The Stage-7 distillation regime addresses reasoning collapse during long-horizon proofs and complex architectural refactors:

1. Recursive Step-Verification: The student model is trained on Claude Opus 4.6 thinking scratchpads where intermediate assertions are continually cross-checked against formalized mathematical constraints before moving to subsequent deductions.
2. Backtracking Recovery Trajectories: Unlike standard SFT models that hallucinate once an erroneous assumption is generated, Stage-7 traces contain explicit error-detection and self-correction tokens (Wait, let me recalculate that...), allowing the model to recover gracefully from logic branches.
3. Q8_0 Quantization Preservation: At 8-bit precision, weight outlier activations in early MLP expansion layers are preserved with 99.95% FP16 fidelity, ensuring the nuanced CoT formatting of Claude Opus 4.6 is preserved without token skipping or stylistic degradation.

---

Technical Specifications

| Architectural Parameter | Verified Specification |

| :--- | :--- |

| Base Foundation Architecture | Gemma 4 31B Dense Transformer |

| Distillation Teacher | Anthropic Claude Opus 4.6 (Extended Thinking Traces) |

| Distillation Checkpoint Stage | Stage-7 (s7) High-Convergence Checkpoint |

| Quantization Scheme | GGUF Q8_0 (8-Bit Symmetric Block Quantization) |

| Primary Checkpoint File | gemma-4-31b-claude-4.6-opus-thinking-distilled-s7-q8_0.gguf |

| Exact Checkpoint File Size | 32.64 GB (32,635,658,528 bytes) |

| Total Parameters | 31.4 Billion (Dense) |

| Extended Context Window | 131,072 Tokens (131K) |

| Primary Execution Runtime | Anvil Engine (Solstice Labs) |

| Compatible Execution Engines | llama.cpp (b3600+), Ollama, LM Studio |

| Hardware Backends | Apple Silicon Metal (TurboFlash), NVIDIA CUDA, Vulkan, HIP |

---

Hardware Sizing & Memory Footprint

| Target Hardware | Total RAM / VRAM | Execution Setup | Expected Generation Speed |

| :--- | :---: | :--- | :---: |

| Apple Mac Studio (M2/M3/M4 - 64GB Unified) | 64 GB Unified | Anvil Metal TurboFlash | 22–32 tok/s |

| Apple MacBook Pro (36GB–48GB Unified) | 48 GB Unified | Anvil (turbo3 KV Cache) | 18–26 tok/s |

| NVIDIA RTX 4090 / 3090 (24GB VRAM) | 24GB VRAM + 32GB RAM | Partial GPU Offload (38/48 layers) | 10–15 tok/s |

| Dual NVIDIA RTX 3090 / 4090 (48GB VRAM) | 48 GB VRAM | Full GPU Offload (CUDA) | 30–42 tok/s |

| Enterprise NVIDIA A100 / H100 (80GB) | 80 GB VRAM | Anvil Server / Full Tensor Cores | 65+ tok/s |

---

Quickstart Guide

Option 1: Primary Execution with Anvil Engine (Recommended)

Anvil provides native memory paging, Google TurboQuant KV compression, and seamless single-command CLI execution:

# 1. Install Anvil CLI
curl -fsSL https://anvil-llm.github.io/anvil/install.sh | sh

# 2. Pull the model checkpoint directly from Hugging Face
anvil pull hf:Solstice-AI/gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF

# 3. Launch an interactive session with TurboQuant 3-bit KV compression
anvil run hf:Solstice-AI/gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF \
  --ctx 131072 \
  --type-k turbo4 \
  --type-v turbo3

# 4. Host a high-concurrency OpenAI-compatible API endpoint
anvil serve hf:Solstice-AI/gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF \
  --port 8080 \
  --host 0.0.0.0 \
  --ctx 131072

---

Option 2: Direct Execution via llama.cpp

# Direct execution streaming from Hugging Face Hub using llama-cli
# (Option A: Interactive conversation mode using model\'s embedded chat template)
llama-cli \
  --hf-repo Solstice-AI/gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF \
  --hf-file gemma-4-31b-claude-4.6-opus-thinking-distilled-s7-q8_0.gguf \
  -cnv \
  -ngl 99 \
  -fa \
  -ctk q4_0 \
  -ctv q4_0 \
  -c 32768

# (Option B: Single-prompt batch inference)
llama-cli \
  --hf-repo Solstice-AI/gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF \
  --hf-file gemma-4-31b-claude-4.6-opus-thinking-distilled-s7-q8_0.gguf \
  -p "<start_of_turn>user
Explain why Gödel's incompleteness theorems fundamentally altered mathematical logic.<end_of_turn>
<start_of_turn>model
<thought>
" \
  -ngl 99 \
  -fa \
  -ctk q4_0 \
  -ctv q4_0 \
  -c 32768 \
  -n 1024

# Launch persistent inference server
llama-server \
  --hf-repo Solstice-AI/gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF \
  --hf-file gemma-4-31b-claude-4.6-opus-thinking-distilled-s7-q8_0.gguf \
  --port 8080 \
  -ngl 99 \
  -fa \
  -ctk q4_0 \
  -ctv q4_0 \
  -c 65536

---

Option 3: Manual Download via modern hf CLI

hf download Solstice-AI/gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF \
  gemma-4-31b-claude-4.6-opus-thinking-distilled-s7-q8_0.gguf \
  --local-dir .

---

Chat Template & Prompt Schema

This model uses the Gemma turn-based prompt structure with explicit <thought>...</thought> scratchpads for Stage-7 Claude-distilled reasoning traces:

Reasoning Prompt Format

<start_of_turn>user
{USER_PROMPT}<end_of_turn>
<start_of_turn>model
<thought>
{INTERNAL_REASONING_CHAIN_AND_STEP_VERIFICATION}
</thought>

{FINAL_ANSWER_OR_CODE}<end_of_turn>

Python Tokenizer Automation

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Solstice-AI/Solstice-AI__gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF")
messages = [
    {"role": "user", "content": "Prove that the square root of 2 is irrational."}
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

---

Citation & Acknowledgments

@software{solstice2026_gemma4_claude_opus_s7,
  title={gemma-4-31b-claude-opus-4.6-thinking-distilled-s7: Dense Stage-7 Reasoning Quantization},
  author={Solstice-AI Research Team},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/Solstice-AI/gemma-4-31b-claude-opus-4.6-thinking-distilled-s7-GGUF}
}

We gratefully acknowledge:

• Anthropic for pioneering the deep recursive thinking paradigms in Claude Opus 4.6.
• The Google Gemma Team for the foundational Gemma 31B architectural dense weights.
• The Solstice Labs Infrastructure Team for developing the Anvil execution engine and precision GGUF quantization pipelines.

---

<p align="center">

<b>Solstice-AI</b> &bull; Frontier AI for everyone, everywhere. &bull; <a href="https://solstice-ai.co">solstice-ai.co</a> &bull; <a href="https://github.com/Solstice-Labs/anvil">Anvil Runtime</a>

</p>