---

license: cc-by-nc-4.0

base_model: pmrccs/qwen3-1.7b-tool-calling-v4

pipeline_tag: text-generation

language:

• en

tags:

• gguf
• qwen3
• tool-calling
• function-calling
• agentic
• quantized
• llama.cpp
• ggml

datasets:

• NousResearch/hermes-function-calling-v1
• MadeAgents/xlam-irrelevance-7.5k

library_name: gguf

---

Qwen3-1.7B Tool-Calling v4 — GGUF

A quantized GGUF version of pmrccs/qwen3-1.7b-tool-calling-v4, a fine-tuned Qwen3-1.7B model designed for function calling and tool-use tasks.

This repository makes the fine-tuned model available in GGUF format for local inference with compatible runtimes, including llama.cpp and supported desktop applications.

Model Overview

| Property | Details |

|---|---|

| Architecture | Qwen3-1.7B |

| Original base model | unsloth/Qwen3-1.7B |

| Fine-tuned model | Qwen3-1.7B Tool-Calling v4 |

| Format | GGUF |

| Primary purpose | Tool calling and function selection |

| Language | English |

| Context length | Up to 32,768 tokens, subject to runtime configuration and available memory |

| License | CC-BY-NC-4.0 |

This is a quantized distribution of the existing v4 fine-tune, not a separately trained model. Quantization reduces storage and memory requirements, with potential trade-offs in output quality depending on the quantization method.

What Is This Model Designed For?

The model was fine-tuned to improve its ability to select and invoke functions based on user requests and available tool definitions.

Intended use cases include:

• Selecting an appropriate function from a small set of candidate tools.
• Generating function-call arguments from natural-language instructions.
• Integrating language models into lightweight agents and automation workflows.
• Running tool-calling models locally on consumer hardware.
• Experimenting with structured function calling in resource-constrained environments.

Training Background

The original v4 model was fine-tuned from unsloth/Qwen3-1.7B using Unsloth, TRL, and PyTorch distributed data parallel training.

The original training setup used two NVIDIA Tesla T4 GPUs, LoRA with rank 16, and an effective batch size of 32. Training ran for 300 steps.

The original fine-tuning datasets included:

• NousResearch/hermes-function-calling-v1
• MadeAgents/xlam-irrelevance-7.5k

For more information about the training process and the original model, see the v4 model card.

Quantization

GGUF is a model file format commonly used for local inference. Quantization can reduce the memory footprint and make inference more accessible on systems with limited RAM or VRAM.

The available quantization files and their sizes are listed in the Files and versions section.

Choose a quantization compatible with your available memory and desired inference quality. Different quantization levels can produce different results, so the original model's benchmark scores should not automatically be assumed to apply identically to every GGUF variant.

Usage

Option 1: Run with llama.cpp

Install or build llama.cpp with support for your hardware.

Download a GGUF file from this repository, then run:

llama-cli \
  -m ./your-model-file.gguf \
  -cnv \
  -n 512

Replace your-model-file.gguf with the actual filename you downloaded.

The -cnv option enables conversation mode. Adjust the generation length and other settings to suit your application.

Option 2: Use a local GUI

You can use a compatible local inference application that supports GGUF models, such as:

• LM Studio
• GPT4All
• KoboldCpp

Import or load the GGUF file using the application's model-loading interface. Support for Qwen3 and its chat template depends on the application version.

Option 3: Use Hugging Face to download a model file

Install the Hugging Face CLI:

pip install -U huggingface_hub

Download the repository:

hf download pmrccs/qwen3-1.7b-tool-calling-v4-gguf \
  --local-dir ./qwen3-1.7b-tool-calling-v4-gguf

Then select the appropriate .gguf file for your inference runtime.

Tool-Calling Format

The original v4 fine-tune was trained to produce tool calls using Qwen3-style chat formatting and the following markers:

• <tool_call>
• </tool_call>
• <tool_response>
• </tool_response>

A tool call is a structured model output, not an executed function. Your application or inference framework must parse the output, validate the arguments, execute the requested function, and return its result to the model when appropriate.

Important: Generating tool-call text is not the same as supporting automatic function execution. Actual tool-calling integration depends on the inference runtime, its chat-template configuration, and the application's tool-execution implementation. Verify that your selected runtime preserves the model's expected formatting.

Evaluation

The original v4 model was evaluated locally on the following benchmarks:

| Benchmark | Original v4 result |

|---|---:|

| BFCL v4 — Non-Live AST | 72.56% |

| BFCL v4 — Live AST | 58.62% |

| NexusRaven — Strict F1 | 0.8485 |

| NexusRaven — Lenient F1 | 0.8764 |

These results refer to the original fine-tuned model, not necessarily to the quantized GGUF files. Quantization can affect function selection, argument generation, and structured-output reliability.

The benchmark runs were performed locally and were not submitted to a public leaderboard. See the evaluation artifacts repository for additional details.

Known Limitations

The original fine-tune has documented limitations:

• Parallel tool calls: The training setup was designed around single-call behavior. Sequential tool execution is a more appropriate starting point.
• Java and JavaScript schemas: These were not represented in the training data. Python-style function schemas are the intended starting point.
• No-call decisions: The model may express a refusal in natural language rather than producing a dedicated no-call sentinel.
• Strict enumerations: Generated values may need validation or normalization against allowed enum values.
• Persistent state: Cross-session memory is not implemented by the model itself.

Quantization may introduce additional quality changes. Validate outputs before using them in production workflows.

Intended Use and Safety

This model is intended for research, experimentation, local inference, and development of tool-using applications.

Do not execute generated function calls without validating the selected function and its arguments. Apply appropriate permission checks and safeguards before allowing model outputs to trigger external actions.

License and Attribution

This repository is distributed under the Creative Commons Attribution-NonCommercial 4.0 International (CC-BY-NC-4.0) license, as indicated by the original model's published license.

• Original fine-tuned model: pmrccs/qwen3-1.7b-tool-calling-v4
• Original base model: unsloth/Qwen3-1.7B

Review the applicable licenses and attribution requirements of the original model and its upstream components before redistribution or commercial use.

---

Maintainer: pmrccs