---

language:

- en

- zh

- fr

- es

- pt

- de

- it

- ru

- ja

- ko

- ar

license: other

license_name: qwen-research

license_link: https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE

base_model: Qwen/Qwen2.5-3B-Instruct

tags:

- gguf

- quantized

- q4_k_m

- qwen

- qwen2.5

- instruct

- llama-cpp

- edge-deployment

- structured-output

- pbh-applied-systems

- quant-eval

---

Qwen2.5-3B-Instruct · GGUF Q4\_K\_M

Quantized, converted, and evaluated by PBH Applied Systems, LLC

— Applied AI/ML Consulting · LLM Optimization & Deployment · Quantized AI Infrastructure

> 🔬 This repository is part of a production-oriented evaluation series. Every model published under pbhappliedsystems has been independently evaluated using quant_eval v7.21 — a proprietary behavioral evaluation harness developed by PBH Applied Systems. Scores measure real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning families — not perplexity or benchmark leaderboard proxies.

> ⚖️ License Notice: This model is governed by the Qwen Research License, which permits non-commercial use only. Commercial use requires a separate license from Alibaba Cloud. See the License section for full details.

---

Try This Model in the Live AI Agent Demo

Launch the PBH Applied Systems AI Agent Demo →

This model is part of the PBH Applied Systems live AI Agent Demo, where visitors can test evaluated quantized open-weight models across production-style agent workflows: reasoning and analysis, document intelligence, and code automation.

The demo uses quant_eval results to show how model selection changes by task. A model that performs well for long-context document analysis may not be the best choice for hard multi-step planning, strict tool-use workflows, or production code generation. Each deployed model is evaluated for practical agent behavior, including coherence, instruction following, reasoning, task completion, structured output reliability, tool-use behavior, and quantization impact.

For this repository, the Q4_K_M variant represents the deployment-focused model: smaller, faster, and more cost-efficient than the F16 baseline. The evaluation results below explain where this quantized model preserves useful behavior, where quantization introduces risk, and what guardrails are recommended before production deployment.

The purpose of the demo is simple: let prospects test the same kind of evaluated quantized models that PBH Applied Systems deploys for real agentic AI systems.

---

Model Description

This repository contains the 4-bit quantized (Q4\_K\_M) GGUF of Qwen/Qwen2.5-3B-Instruct, a 3-billion parameter instruction-tuned model from Alibaba Cloud (September 2024 release). Qwen2.5-3B-Instruct is the smallest model in the PBH Applied Systems evaluated series, and at Q4\_K\_M precision it represents the most hardware-accessible deployment option — requiring as little as 4 GB VRAM and fitting on consumer edge hardware.

The full-precision F16 baseline is published separately at pbhappliedsystems/qwen-2.5-3B-instruct-gguf-F16.

Key Characteristics

• Parameters: 3B
• Format: GGUF Q4\_K\_M
• File size: 1.93 GB
• SHA256: 9ab3bc9beaddaec3700d5cc754b52e1501a3fd172bc7fc3ee3eb8e1d388ee043
• Minimum VRAM (GPU inference): ~4 GB
• Recommended GPU tier: Any CUDA-capable GPU · CPU inference viable
• Context window: 32,768 tokens
• Inference speed (eval hardware): avg 0.390 sec/case on RTX 4090 — fastest Q4\_K\_M in the evaluated series
• License: Qwen Research License (non-commercial)

---

PBH Applied Systems Evaluation — quant\_eval v7.21

> Evaluation conducted by PBH Applied Systems, LLC using quant_eval v7.21

> Run ID: 20260221_041137 · Fixtures: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c...) · Seed: 42

> Hardware: NVIDIA RTX 4090 · Total rows evaluated: 84 (42 F16 · 42 Q4\_K\_M)

Aggregate Scores (Q4\_K\_M)

Scores are normalized to [0.0 – 1.0]. Higher is better.

| Dimension | Score | Series Context |

|---|---:|---|

| Task Completion | 0.4905 | Expected at 3B scale |

| Reasoning | 0.3704 | Limited by parameter count |

| Coherence | 0.9074 | Strong at 3B |

| Instruction Following | 0.6599 | Moderate |

| Avg inference time | 0.390 sec/case | Fastest in series |

Per-Family Pass Rates

F16 Baseline (full_weight_transformers)

| Family | N | Pass Rate | Avg Secs | Notes |

|---|---:|---:|---:|---|

| json\_multistep | 5 | 0.000 | 3.704 | checks\_consistent\_ok bottleneck |

| stateful\_followup | 2 | 1.000 | 0.525 | Both turns exact match |

| toolcall\_only | 2 | 0.000 | 0.405 | tool\_name\_ok=1, args\_ok=0 |

| mixed\_brief\_json | 2 | 0.000 | 0.695 | JSON valid; ANSWER line absent — see note |

| toolcall | 2 | 1.000 | 0.745 | Stage-1 passes; final\_mismatch — see note |

| json | 4 | n/a | 2.072 | bucket\_score avg = 10.000 |

| fuzz | 20 | n/a | 1.437 | bucket\_score avg = 7.500 — 15/20 pass |

| mcq | 5 | n/a | 0.022 | Empty raw output on all 5 |

Q4\_K\_M (quantized_llama_cpp)

| Family | N | Pass Rate | Δ vs F16 | Avg Secs | Notes |

|---|---:|---:|---:|---:|---|

| json\_multistep | 5 | 0.200 | +0.200 | 0.985 | ms\_easy\_01 recovers |

| stateful\_followup | 2 | 1.000 | 0.000 | 0.125 | Perfect retention |

| toolcall\_only | 2 | 0.000 | 0.000 | 0.100 | Wrong arg key names |

| mixed\_brief\_json | 2 | 1.000 | +1.000 | 0.155 | Full recovery at Q4\_K\_M |

| toolcall | 2 | 1.000 | 0.000 | 0.215 | Stage-1 passes; final\_mismatch — see note |

| json | 4 | n/a | — | 0.510 | bucket\_score avg = 10.000 |

| fuzz | 20 | n/a | — | 0.408 | bucket\_score avg = 7.500 — same 15/20 |

| mcq | 5 | n/a | — | 0.010 | 3/5 pass at Q4\_K\_M |

---

Key Findings

Finding 1: The Speed Story — Fastest Model in the Evaluated Series

At 0.390 sec/case average, this Q4\_K\_M variant is the fastest model evaluated in the PBH Applied Systems series. Individual family timings:

| Family | Q4\_K\_M Avg Secs |

|---|---:|

| mcq | 0.010 |

| stateful\_followup | 0.125 |

| toolcall\_only | 0.100 |

| mixed\_brief\_json | 0.155 |

| toolcall | 0.215 |

| fuzz | 0.408 |

| json | 0.510 |

| json\_multistep | 0.985 |

MCQ responses at 10 milliseconds. Stateful follow-up at 125 milliseconds. Even the most complex planning cases complete in under one second. At 1.93 GB, this model runs on hardware where no other model in this series fits — including systems without dedicated GPUs.

This speed profile is the primary deployment argument for Qwen2.5-3B Q4\_K\_M. It is not the most capable model evaluated. It is the most accessible model evaluated, with a capability ceiling that is clearly defined by the evaluation data below.

Finding 2: checks\_consistent\_ok = 0.200 — The 3B Ceiling

The single most consistent failure signal across both runners and all json\_multistep cases is checks_consistent_ok. Both runners achieve the same rate (0.200) — 1/5 cases pass. This is the signal that measures whether the model's intermediate reasoning steps are internally self-consistent.

This is a parameter-count limitation, not a quantization artifact. The same 5 fuzz cases fail on both runners (fuzz_0001, _0002, _0003, _0010, _0012) at the same bucket scores, confirming the failure boundary is set by the model's capacity, not by the precision level. A 3B model that can produce valid JSON schemas reliably but cannot maintain consistent multi-step reasoning chains is behaving exactly as expected at this scale.

| Signal | F16 Rate | Q4\_K\_M Rate |

|---|---:|---:|

| schema\_ok | 0.800 | 1.000 |

| checks\_consistent\_ok | 0.200 | 0.200 |

| stop\_semantics\_ok | 0.200 | 0.400 |

| oracle\_equiv\_ok | 0.200 | 0.400 |

Q4\_K\_M actually improves over F16 on schema_ok (0.800 → 1.000), stop_semantics_ok (0.200 → 0.400), and oracle_equiv_ok (0.200 → 0.400). The checks_consistent_ok signal, which is the deepest reasoning consistency test, holds at 0.200 regardless of precision.

Finding 3: F16 Role-Token Contamination

The F16 evaluation reveals a specific issue with how the HuggingFace Transformers runner handles this model's chat template. Raw outputs for several families show truncated role token prefixes:

• toolcall cases: ician {"tool_name": "add", ...} — the role token "technician" is truncated to "ician" and appears in the response body
• stateful cases: ician {"counter": 2} — same prefix contamination
• mixed_brief_json: user ANSWER: 13 {...} — the "user" role token appears as literal text

The JSON content in these cases is correct, but the role prefix causes extraction failures. This explains why toolcall shows final_mismatch at F16 (the stage-1 JSON is valid but the post-response text includes role tokens), and why mixed_brief_json fails answer_line_ok at F16 despite having json_parse_ok=1 and schema_ok=1.

This is a runner configuration issue, not a model quality issue. The Q4_K_M llama.cpp runner does not exhibit this behavior.

Finding 4: F16 ms\_easy\_01 — Wrong Output Type

ms_easy_01 at F16 is the only case in the evaluated series where a model returns an entirely wrong output type for json_multistep. The F16 model produces:

[{"shelf": "A", "item": "P", "can_place": 1}]

An array, where the task requires a specific schema object with plan, checks, and final keys. This causes schema_ok=0 — the only F16 json_multistep case with a schema failure. At Q4\_K\_M, ms_easy_01 recovers: schema_ok=1, oracle_equiv_ok=1, checks_consistent_ok=1 — a clean pass.

Finding 5: toolcall\_only — Schema Vocabulary Errors

Both toolcall_only cases at Q4\_K\_M show a consistent vocabulary mismatch:

| Case | Q4\_K\_M Raw Output | Expected |

|---|---|---|

| toolonly\_01 | {"tool": "add", "operands": [5, 10]} | {"tool_name": "add", "args": {"a": 5, "b": 10}} |

| toolonly\_02 | {"tool": "add", "operands": [25, 75]} | {"tool_name": "add", "args": {"a": 25, "b": 75}} |

The model uses "tool" instead of "tool_name" and "operands" (array) instead of "args" (object with named keys). Tool name recognition passes (tool_name_ok=1) because the extractor finds "add" in the payload. Argument extraction fails because "operands": [5, 10] is not a valid schema for {"a": integer, "b": integer}.

This is a schema vocabulary issue at 3B scale — the model knows the tool concept but not the exact field name schema. A system prompt that explicitly states the required key names (tool_name, args.a, args.b) would likely resolve this for basic tools.

Finding 6: toolcall — Correct Arithmetic, EOS Contamination

Both toolcall cases at Q4\_K\_M pass stage-1 (tool dispatch valid) but produce final_mismatch:

| Case | Raw Output | Expected |

|---|---|---|

| tool\_01 | {...add(2,3)...}<\|im_end\|> 5<\|im_end\|> | 5 |

| tool\_02 | {...add(10,-4)...}<\|im_end\|> 6<\|im_end\|> | 6 |

The arithmetic is correct. The EOS token (<|im_end|>) appended to the answer string causes the string comparison to fail. A re.sub(r'<\|im_end\|>', '', raw) strip resolves it. See the Usage section for the validated extraction pattern.

Finding 7: MCQ — A-Bias Pattern

| Case | Q4\_K\_M Result | Raw |

|---|---|---|

| mcq\_01 | ✅ PASS | B |

| mcq\_02 | ❌ FAIL | A (wrong) |

| mcq\_03 | ✅ PASS | C |

| mcq\_04 | ✅ PASS | B |

| mcq\_05 | ❌ FAIL | A (wrong) |

The two failures both produce A. The same A-bias pattern observed in Mistral-Nemo appears here at 3B scale. At F16, all 5 MCQ cases produce empty raw output (raw=''), suggesting the F16 runner cannot extract the choice from the model's response format at this parameter count.

---

Signal-Level Diagnostics

Q4\_K\_M — json\_multistep

| Signal | F16 Rate | Q4\_K\_M Rate | Delta |

|---|---:|---:|---:|

| schema\_ok | 0.800 | 1.000 | +0.200 |

| checks\_consistent\_ok | 0.200 | 0.200 | 0.000 |

| stop\_semantics\_ok | 0.200 | 0.400 | +0.200 |

| oracle\_equiv\_ok | 0.200 | 0.400 | +0.200 |

Q4\_K\_M — stateful\_followup

| Signal | Rate |

|---|---:|

| turn1\_parse\_ok | 1.000 |

| turn2\_parse\_ok | 1.000 |

| turn1\_exact\_match | 1.000 |

| turn2\_exact\_match | 1.000 |

Q4\_K\_M — toolcall\_only

| Signal | Rate |

|---|---:|

| tool\_name\_ok | 1.000 |

| args\_ok | 0.000 |

Q4\_K\_M — mixed\_brief\_json

| Signal | Rate |

|---|---:|

| answer\_line\_ok | 1.000 |

| json\_parse\_ok | 1.000 |

| schema\_ok | 1.000 |

---

Recommended Use Cases

✅ Deploy with Confidence (Q4\_K\_M, Non-Commercial)

• Stateful multi-turn agents — Perfect two-turn retention (1.000) at 0.125 sec/case. The fastest stateful performance in the evaluated series.
• Structured JSON outputs (single-step) — json bucket\_score 10.000, fuzz bucket\_score 7.500. Fast and reliable for constraint-adherent single-step placements.
• Hybrid brief + JSON responses — mixed_brief_json passes at 1.000 in 0.155 sec. Clean ANSWER line + valid JSON.
• Edge and resource-constrained deployment — 1.93 GB at Q4\_K\_M. Runs on 4 GB VRAM, embedded GPUs, and CPU-only environments where no other evaluated model fits.
• High-throughput batch processing — At 0.390 sec/case average, this model can process thousands of structured inference tasks per hour on modest hardware.

⚠️ Use with Guardrails (Q4\_K\_M)

• Scaffolded tool-calling — toolcall stage-1 passes at 1.000 but add EOS stripping before final answer extraction. Arithmetic results are correct.
• Bare tool-call dispatch — toolcall_only fails on args schema ("operands" vs "args"). Provide explicit schema examples in the system prompt or add a normalization layer.
• Multi-step planning (easy difficulty only) — ms\_easy\_01 passes. Higher difficulties fail consistently on checks_consistent_ok. Use with external validation.
• MCQ applications — 3/5 pass but A-bias exists. Add chain-of-thought prompting or validation for MCQ pipelines.

❌ Not Recommended (Q4\_K\_M)

• Medium-to-hard multi-step planning — ms\_med and ms\_hard cases fail on internal consistency. The 3B parameter count is the limiting factor, not the quantization.
• Any commercial use without a Qwen commercial license — The Qwen Research License restricts use to non-commercial research and evaluation purposes. Contact Alibaba Cloud for commercial licensing.

---

Hardware Requirements

| Configuration | VRAM Required | Notes |

|---|---|---|

| Q4\_K\_M (this repo) · GPU | ~4 GB | Fits on embedded/mobile GPUs |

| Q4\_K\_M · CPU only | 4 GB RAM | Viable; slower inference |

| Q4\_K\_M · 32K context | ~8 GB | Full context window |

| F16 (companion repo) · GPU | ~8 GB | 6.18 GB model + overhead |

---

Usage

Installation

pip install llama-cpp-python huggingface_hub

For GPU acceleration (CUDA):

CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall --no-cache-dir

Python — llama-cpp-python

from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/qwen-2.5-3B-instruct-gguf-Q4-K-M",
    filename="qwen-2.5-3B-instruct-gguf-Q4-K-M.gguf"
)

llm = Llama(
    model_path=model_path,
    n_ctx=4096,
    n_gpu_layers=-1,  # -1 for full GPU; set 0 for CPU-only
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a precise assistant. Return structured JSON when asked. Use exactly the key names specified."
        },
        {
            "role": "user",
            "content": "Summarize the following and return JSON with keys: summary, sentiment, action_items."
        }
    ],
    temperature=0.15,
    max_tokens=512,
)

print(response["choices"][0]["message"]["content"])

For CPU-only deployment (no GPU required):

llm = Llama(
    model_path=model_path,
    n_ctx=2048,
    n_gpu_layers=0,    # CPU-only
    n_threads=8,       # Tune to your CPU core count
    verbose=False,
)

For tool-calling with EOS stripping (addresses toolcall final_mismatch finding):

import json, re
from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/qwen-2.5-3B-instruct-gguf-Q4-K-M",
    filename="qwen-2.5-3B-instruct-gguf-Q4-K-M.gguf"
)

llm = Llama(model_path=model_path, n_ctx=2048, n_gpu_layers=-1, verbose=False)

def call_tool_with_cleanup(prompt: str) -> dict:
    """
    Tool dispatch with EOS stripping.
    quant_eval v7.21: toolcall stage-1 pass=1.000; final_mismatch due to <|im_end|> suffix.
    Arithmetic is correct — strip EOS before downstream processing.
    """
    response = llm.create_chat_completion(
        messages=[
            {
                "role": "system",
                "content": (
                    "You are a tool-calling assistant. "
                    "When calling a tool, output JSON as: "
                    '{"tool_name": "<name>", "args": {"a": <n>, "b": <n>}}\n'
                    "Then on the next line, output the result as a plain number."
                )
            },
            {"role": "user", "content": prompt}
        ],
        temperature=0.0,
        max_tokens=128,
    )
    raw = response["choices"][0]["message"]["content"]
    # Strip EOS tokens before processing
    clean = re.sub(r'<\|im_end\|>', '', raw).strip()
    return {"raw": raw, "clean": clean}

result = call_tool_with_cleanup("Use the add tool to compute 5 plus 10.")
print(result["clean"])

CLI — llama-cli

llama-cli \
  --model qwen-2.5-3B-instruct-gguf-Q4-K-M.gguf \
  --chat-template qwen2 \
  --system-prompt "You are a precise assistant. Use exactly the key names specified in any JSON schema." \
  --prompt "Return a JSON object with keys: summary, risk_level, action_items." \
  --n-predict 512 \
  --ctx-size 4096 \
  --n-gpu-layers -1 \
  --temp 0.15

For server deployment:

llama-server \
  --model qwen-2.5-3B-instruct-gguf-Q4-K-M.gguf \
  --chat-template qwen2 \
  --ctx-size 4096 \
  --n-gpu-layers -1 \
  --port 8080 \
  --host 0.0.0.0

Query via the OpenAI-compatible API:

from openai import OpenAI
import re

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-required")

response = client.chat.completions.create(
    model="qwen-2.5-3B-instruct-gguf-Q4-K-M",
    messages=[{"role": "user", "content": "Your prompt here"}],
    temperature=0.15,
)
# Strip EOS tokens from output
clean = re.sub(r'<\|im_end\|>', '', response.choices[0].message.content).strip()
print(clean)

---

Evaluation Artifacts

The full per-case evaluation CSV (comparison_results_v7_21_Qwen2.5_3B_Instruct_20260221_041137.csv) and rollup.json are published in this repository for independent verification.

---

Artifact Provenance

| Artifact | Format | Size | SHA256 |

|---|---|---|---|

| qwen-2.5-3B-instruct-gguf-Q4-K-M.gguf | GGUF Q4\_K\_M | 1.93 GB | 9ab3bc9beaddaec3700d5cc754b52e1501a3fd172bc7fc3ee3eb8e1d388ee043 |

| F16 (companion repo) | GGUF F16 | 6.18 GB | 65a0239fc9f9a40e2d4f79ae5e158cad423c2476fe089c744d5e6a6ff6fc9330 |

Both artifacts were produced from Qwen/Qwen2.5-3B-Instruct using a custom-built llama.cpp conversion and quantization pipeline developed by PBH Applied Systems.

---

Evaluation Methodology

quant_eval v7.21 is a proprietary behavioral evaluation harness developed by PBH Applied Systems.

Fixture set: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c079371b02a94f3c149eb78a6adc03dc16ff6833b964fbf4174f0)

| Family | Description | Pass Signals |

|---|---|---|

| fuzz | Property-based regression; structured placement correctness | schema\_ok, constraints\_ok |

| json | Single-step structured JSON with constraint rules | schema\_ok, constraints\_ok |

| json_multistep | Multi-step planning with self-check and oracle verification | schema\_ok, checks\_consistent\_ok, stop\_semantics\_ok, oracle\_equiv\_ok |

| mcq | Multiple-choice extraction | choice\_ok |

| stateful_followup | Two-turn state tracking; turn-2 correct given turn-1 | turn1/2\_parse\_ok, turn1/2\_exact\_match |

| mixed_brief_json | Hybrid: natural language answer + valid JSON block | answer\_line\_ok, json\_parse\_ok, schema\_ok |

| toolcall | Tool call embedded in response; parse + schema validation | stage1\_tool\_parse\_ok, stage1\_tool\_schema\_ok |

| toolcall_only | Bare schema-only tool call; strict tool name + args check | tool\_name\_ok, args\_ok |

Evaluation hardware: NVIDIA RTX 4090 (24 GB VRAM)

Evaluation date: February 21, 2026

quant_eval seed: 42

---

🔬 About quant_eval & This Evaluation Series

quant_eval is a proprietary behavioral evaluation harness developed by PBH Applied Systems, LLC. It measures real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning — not perplexity or leaderboard proxies. Every model published under pbhappliedsystems has been independently evaluated using quant_eval before being recommended for any production role.

See it in action: Live AI Agent Demo →

The demo runs production-style agent workflows powered by open-weight models selected through the quant_eval evaluation pipeline.

> Need a deployment recommendation?

> Not sure which quantization level is right for your hardware, latency target, or agent type?

> → pbhappliedsystems.com

---

Evaluated and published by PBH Applied Systems, LLC · patrick@pbhappliedsystems.com

---

About PBH Applied Systems

PBH Applied Systems, LLC is an Oklahoma City–based applied machine learning and AI systems company specializing in production-grade model evaluation, quantization pipelines, agentic AI infrastructure, and scalable AI-driven application development. The organization operates with a strong emphasis on engineering rigor, reproducibility, and real-world deployment constraints — particularly in environments where performance, cost efficiency, and reliability must be balanced against available hardware and budget.

Founder & Principal AI/ML Systems Architect — Patrick Hill

PBH Applied Systems was founded by Patrick Hill, a Data Scientist and AI/ML Engineer with 10+ years of experience delivering advanced analytics, predictive modeling, and decision-support solutions across high-stakes operational environments. Patrick holds a Master of Science in Software Engineering with concentrations in Artificial Intelligence and Machine Learning and a B.S. in Business Finance.

Technical expertise spans:

• Languages & Data: Python, SQL, Linux, Pandas, NumPy, scikit-learn
• ML & Modeling: Supervised and unsupervised learning, neural networks, NLP, transformers, regression, classification, forecasting, and feature engineering
• AI/ML Frameworks: PyTorch, TensorFlow/Keras, HuggingFace Transformers, GGUF, llama.cpp, BitsAndBytes, PEFT, QLoRA
• Deployment & MLOps: Flask APIs, Docker, CI/CD pipelines, REST endpoints, streaming inference, version control
• Data Platforms: Jupyter, Databricks, Power BI, Matplotlib
• Quantization: GGUF conversion, Q4\_K\_M / Q5\_K\_M / Q8\_0 strategies, adapter-per-model evaluation architecture

Published Author

Patrick is the author of Applied Machine Learning: Concepts, Tools, and Case Studies — a 1,200+ page practitioner-oriented textbook adopted as required reading for CSC 373 – Machine Learning at the University of Advancing Technology.

Core Service Areas

1. LLM Optimization & Deployment · 2. AI Evaluation Frameworks · 3. Agentic AI Infrastructure · 4. Scalable AI Application Development · 5. ML Pipeline Design & Analytics · 6. Model & Agent Cataloging

---

📞 Work With PBH Applied Systems

The Qwen2.5-3B Q4\_K\_M evaluation tells a clear story about what to expect at the 3B scale: sub-second inference, reliable structured outputs, solid stateful retention, and a well-defined ceiling on multi-step reasoning consistency. Every finding is verifiable — download the CSV and check the checks_consistent_ok signal yourself.

👉 Book a Scoping Call — Discuss model selection, edge deployment strategy, or evaluation needs directly with Patrick.

👉 Request an Evaluation Report — Full quant_eval behavioral audit for your target model(s). Engagements from $2,500.

Connect

| | |

|---|---|

| 🌐 Website | pbhappliedsystems.com |

| 📧 Email | patrick@pbhappliedsystems.com |

| 💼 LinkedIn | PBH Applied Systems, LLC |

| ▶️ YouTube | @pbhappliedsystems |

| 📸 Instagram | @pbhappliedsystems |

| 👍 Facebook | pbhappliedsystems |

---

License

This GGUF repository is governed by the Qwen Research License Agreement, inherited from the base model Qwen/Qwen2.5-3B-Instruct.

Key terms:

• Non-commercial use only — The license grants a non-exclusive, royalty-free license for research or evaluation purposes only
• Commercial use requires a separate license — Contact Alibaba Cloud to request a commercial license
• Redistribution — Permitted with attribution and license copy included
• Attribution requirement — If used to train or improve another AI model, display "Built with Qwen" or "Improved using Qwen" in documentation
• Governing law — People's Republic of China; exclusive jurisdiction in Hangzhou City People's Courts

Full license text: Qwen Research License Agreement

The quant_eval evaluation methodology, fixture set, and scoring framework are proprietary to PBH Applied Systems, LLC and are not included in this repository.

---

GGUF conversion, quantization, and behavioral evaluation performed by PBH Applied Systems, LLC · quant_eval v7.21 · Run ID: 20260221_041137