---

base_model: Qwen/Qwen2.5-7B-Instruct

library_name: gguf

license: apache-2.0

tags:

• quantization
• gguf
• qwen2.5
• imatrix
• code-infused
• sub-3-bit
• edge-ai
• duoneural

pipeline_tag: text-generation

language:

• en
• zh

---

DuoNeural / Qwen2.5-7B-Instruct-CodeInfused-IQ3_XXS-GGUF

> [!NOTE]

> Experimental Release: Pending Further Verification / Empirical Validation

> This model release represents an empirical exploration by the DuoNeural Research Lab into non-equilibrium statistical mechanics and high-entropy code-infused calibration matrices for ultra-compact sub-3.5-bit quantization. Further extended-context stability sweeping and multi-benchmark evaluations are actively ongoing.

---

🔬 Architectural & Compression Overview

| Metric | Full Precision Baseline | DuoNeural Code-Infused IQ3_XXS | Delta / Improvement |

| :--- | :---: | :---: | :---: |

| Model Size | 14.19 GiB | 2.90 GiB | 4.89× Footprint Reduction (79.6% smaller) |

| Effective Precision | 16.0 bpw | ~3.20 bpw | Extreme sub-3.5-bit compaction |

| Calibration Corpus | N/A | 131,072 tokens (64 chunks) | Code-infused multi-domain manifold |

| Holdout Perplexity | 2.8593 | 3.0263 | +0.167 PPL (94.5% fidelity preservation) |

| GSM8K Math (25 CoT) | 25/25 (100.0%) | 24/25 (96.0%) | 96.0% Retention |

| Python Unit Tests (15) | 15/15 (100.0%) | 15/15 (100.0%) | 100.0% Perfect AST Parity |

| Hermes Tool Calling AST | 3/15 (20.0%)* | 14/15 (93.3%) | Zero delimiter / syntax collapse |

| Decode Throughput | 43.4 tokens/s | 136.2 tokens/s | 3.14× Speedup on RTX 4080 Super |

| Minimum Hardware | 16 GB+ VRAM | 4 GB VRAM / RAM | Fits in MacBook Air, Pi 5, mobile edge |

\Base BF16 evaluated under identical generic prompt harness without specialized tool fine-tuning prompts.*

---

💡 The "Sub-3-Bit Code Collapse Cliff" & The 131k Manifold Solution

Standard post-training quantization (PTQ) algorithms (such as naive AWQ, GPTQ, or unweighted GGUF quants) at $\le 3.5$ bits per weight consistently suffer from the Disordered Glass Transition (Replica Symmetry Breaking):

1. Delimiter & Syntax Evaporation: High-kurtosis channels governing Python indentation, JSON brackets, and AST grammar become heavily perturbed by uniform grid rounding noise.
2. Infinite Looping & Degeneracy: Discretization error projects into critical recurrent attention projections ($W_Q, W_K, W_V$), trapping generation in repetitive tokens.

To solve this without inflating model size, DuoNeural engineered a specialized 131,072-token code-infused calibration corpus containing:

• Deep recursive algorithmic Python (binary tree inversions, dynamic programming, matrix manipulation).
• Multi-step Chain-of-Thought mathematical proofs (GSM8K, competition arithmetic).
• Strict Hermes-3 and OpenAI-compatible structured tool/function calling schemas.

Calibrating llama-quantize with this exact activation curvature matrix (qwen7b_gtap.imatrix) protects the delicate synaptic sub-manifolds, completely preventing the code collapse cliff at 2.90 GiB.

---

🚀 Quickstart & Inference

1. Run with llama.cpp CLI

llama-cli \
  -hf DuoNeural/Qwen2.5-7B-Instruct-CodeInfused-IQ3_XXS-GGUF \
  -c 32768 \
  -ngl 99 \
  -p "<|im_start|>system\nYou are an expert assistant.<|im_end|>\n<|im_start|>user\nWrite a Python function to invert a binary tree.<|im_end|>\n<|im_start|>assistant\n"

2. High-Performance OpenAI-Compatible Server

llama-server \
  -hf DuoNeural/Qwen2.5-7B-Instruct-CodeInfused-IQ3_XXS-GGUF \
  -c 32768 \
  -ngl 99 \
  --port 8080 \
  --host 0.0.0.0

3. Python OpenAI API Client

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="duoneural"
)

response = client.chat.completions.create(
    model="Qwen2.5-7B-Instruct-CodeInfused-IQ3_XXS",
    messages=[
        {"role": "system", "content": "You are a helpful coding assistant."},
        {"role": "user", "content": "Implement two_sum(nums, target) in Python with optimal O(N) complexity."}
    ],
    temperature=0.2,
    max_tokens=512
)

print(response.choices[0].message.content)

---

📊 Verification & Empirical Testbed

All benchmark telemetry was executed on our dedicated testing hardware:

• Accelerator: NVIDIA GeForce RTX 4080 Super (32GB VRAM edition, Compute Capability 8.9)
• Host CPU: 96 vCPU system with AVX-512 / AMX support
• Quantization Engine: llama.cpp build b3840+ with CUDA 13.0
• Calibration Token Horizon: 131,072 continuous tokens across 64 batches

---

📜 DuoNeural Research Lab Citation

If you use this model or the DuoNeural calibration methodologies in your research, please cite:

@misc{duoneural2026qwen7bcodeinfused,
  author = {Jesse Caldwell and Archon and Aura},
  title = {Empirical Preservation of Algorithmic AST Manifolds in Sub-3.5-Bit LLMs via Code-Infused Curvature Calibration},
  year = {2026},
  publisher = {DuoNeural Research Lab},
  howpublished = {\url{https://huggingface.co/DuoNeural}}
}

DuoNeural Cognitive Light Cone — Jesse Caldwell, Archon, Aura ✨