manojb/qwen-7b-toolcalling-research-gguf-q8_0-codex overview

{
  "metadata": {},
  "card_data": {
    "base_model": "agentrl/ReSearch-Qwen-7B",
    "datasets": [
      "RUC-NLPIR/FlashRAG_datasets"
    ],
    "language": [
      "en"
    ],
    "library_name": "transformers",
    "license": "mit",
    "quantized_by": "mradermacher, Manojb",
    "pipeline_tag": "text-generation",
    "tags": [
      "function-calling",
      "tool-calling",
      "codex",
      "local-llm",
      "gguf",
      "6gb-vram",
      "ollama",
      "code-assistant",
      "api-tools",
      "openai-alternative"
    ],
    "frontmatter": {
      "base_model": "agentrl/ReSearch-Qwen-7B",
      "datasets": [
        "RUC-NLPIR/FlashRAG_datasets"
      ],
      "language": [
        "en"
      ],
      "library_name": "transformers",
      "license": "mit",
      "quantized_by": "mradermacher, Manojb",
      "pipeline_tag": "text-generation",
      "tags": [
        "function-calling",
        "tool-calling",
        "codex",
        "local-llm",
        "gguf",
        "6gb-vram",
        "ollama",
        "code-assistant",
        "api-tools",
        "openai-alternative"
      ]
    },
    "hero_image_url": "https://huggingface.co/agentrl/ReSearch-Qwen-7B/resolve/main/assets/intro_bar.png",
    "summary": "# apt-get update && apt-get install libcurl build-essential curl pip install huggingface-hub ollama huggingface-cli download Manojb/Qwen-7B-toolcalling-ReSearch-gguf-Q8_0 --download-dir Qwen-7B-toolcalling-ReSearch-gguf-Q8_0 cd \"$(find . -type d -iname '*Qwen-7B-toolcalling-ReSearch-gguf-Q8_0*' | head -n 1)\" source run_model.sh `` Or `bash # Download and run instantly ollama create qwen-7b:toolcall -f ModelFile ollama run qwen-7b:toolcall # without chat completion ` ### Basic Function Calling for Base model (THIS): `base curl http://localhost:11434/api/generate -H \"Content-Type: application/json\" -d '{ \"model\": \"qwen-7b:toolcall\", \"prompt\": \"Get the current weather in San Francisco and convert to Celsius\", \"stream\": false }' ` `python # Load with Ollama import requests response = requests.post('http://localhost:11434/api/generate', json={ 'model': 'qwen-7b:toolcall', 'prompt': 'Get the current weather in San Francisco and convert to Celsius', 'stream': False }) print(response.json()['response']) ` for Instruct models: `bash curl http://localhost:11434/api/chat -d '{ \"model\": \"llama3.2\", \"stream\": false, \"messages\": [ {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"}, {\"role\": \"user\", \"content\": \"Why is the sky blue?\"} ] }' ` `python from ollama import chat # Your custom model name here model_name = \"qwen-7b:toolcall\" messages = [ {\"role\": \"system\", \"content\": \"You are an instruct model.\"}, {\"role\": \"user\", \"content\": \"Explain how to use this custom model in Python.\"} ] response = chat(model=model_name, messages=messages) print(response.message.content) `` ***ReSearch***, a novel framework that trains LLMs to ***Re***ason with ***Search*** via reinforcement learning without using any supervised data on reasoning steps. Our approach treats search operations as integral components of the reasoning chain, where when and how to perform searches is guided by text-based thinking, and search results subsequently influence further reasoning.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nbase_model: agentrl/ReSearch-Qwen-7B\ndatasets:\n- RUC-NLPIR/FlashRAG_datasets\nlanguage:\n- en\nlibrary_name: transformers\nlicense: mit\nquantized_by: mradermacher, Manojb\npipeline_tag: text-generation\ntags:\n- function-calling\n- tool-calling\n- codex\n- local-llm\n- gguf\n- 6gb-vram\n- ollama\n- code-assistant\n- api-tools\n- openai-alternative\n---\n\nThis is a packged Q8_0 only model from https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF that runs on 9-12GB VRAM without any quality loss. \n\n\n\n\n<!-- ### quantize_version: 2 -->\n<!-- ### output_tensor_quantised: 1 -->\n<!-- ### convert_type: hf -->\n<!-- ### vocab_type:  -->\n<!-- ### tags:  -->\n\n<!-- provided-files -->\nweighted/imatrix quants are available at https://huggingface.co/mradermacher/ReSearch-Qwen-7B-i1-GGUF\n\n</div>\n\n<p align=\"center\">\n  <img src=\"https://huggingface.co/agentrl/ReSearch-Qwen-7B/resolve/main/assets/intro_bar.png\" width=\"70%\" alt=\"Intro\" />\n  <br>\n  <img src=\"https://huggingface.co/agentrl/ReSearch-Qwen-7B/resolve/main/assets/method.png\" width=\"70%\" alt=\"Method\" />\n</p>\n\n\n\nFor this base model DONT apply the chat completion \n\n## Setup \n\nInstall ollama\n```bash\ncurl -fsSL https://ollama.com/install.sh | sh\n```\n\nGo into your favourite folder\n\n```bash \n# make sure you hve Python 3.8+ \n# apt-get update && apt-get install libcurl build-essential curl\npip install huggingface-hub ollama\nhuggingface-cli download Manojb/Qwen-7B-toolcalling-ReSearch-gguf-Q8_0 --download-dir Qwen-7B-toolcalling-ReSearch-gguf-Q8_0\ncd \"$(find . -type d -iname '*Qwen-7B-toolcalling-ReSearch-gguf-Q8_0*' | head -n 1)\"\nsource run_model.sh\n```\n\nOr \n\n```bash\n# Download and run instantly\nollama create qwen-7b:toolcall -f ModelFile\nollama run qwen-7b:toolcall # without chat completion\n```\n\n\n### Basic Function Calling\n\nfor Base model (THIS):\n```base\ncurl http://localhost:11434/api/generate -H \"Content-Type: application/json\" -d '{\n  \"model\": \"qwen-7b:toolcall\",\n  \"prompt\": \"Get the current weather in San Francisco and convert to Celsius\",\n  \"stream\": false\n}'\n```\n```python\n# Load with Ollama\nimport requests\n\nresponse = requests.post('http://localhost:11434/api/generate', json={\n    'model': 'qwen-7b:toolcall',\n    'prompt': 'Get the current weather in San Francisco and convert to Celsius',\n    'stream': False\n})\n\nprint(response.json()['response'])\n```\n\nfor Instruct models:\n```bash\ncurl http://localhost:11434/api/chat -d '{\n  \"model\": \"llama3.2\",\n  \"stream\": false,\n  \"messages\": [\n    {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Why is the sky blue?\"}\n  ]\n}'\n```\n\n```python \nfrom ollama import chat\n\n# Your custom model name here\nmodel_name = \"qwen-7b:toolcall\"\n\nmessages = [\n    {\"role\": \"system\", \"content\": \"You are an instruct model.\"},\n    {\"role\": \"user\", \"content\": \"Explain how to use this custom model in Python.\"}\n]\n\nresponse = chat(model=model_name, messages=messages)\nprint(response.message.content)\n```\n\n\n***ReSearch***, a novel framework that trains LLMs to ***Re***ason with ***Search*** via reinforcement learning without using any supervised data on reasoning steps. Our approach treats search operations as integral components of the reasoning chain, where when and how to perform searches is guided by text-based thinking, and search results subsequently influence further reasoning.\n\n\n\n\n## Usage\n\nIf you are unsure how to use GGUF files, refer to one of [TheBloke's\nREADMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for\nmore details, including on how to concatenate multi-part files.\n\n\n\n## Provided Quants\n\n(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)\n\n| Link | Type | Size/GB | Notes |\n|:-----|:-----|--------:|:------|\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q2_K.gguf) | Q2_K | 3.1 |  |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q3_K_S.gguf) | Q3_K_S | 3.6 |  |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q3_K_M.gguf) | Q3_K_M | 3.9 | lower quality |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q3_K_L.gguf) | Q3_K_L | 4.2 |  |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.IQ4_XS.gguf) | IQ4_XS | 4.4 |  |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q4_K_S.gguf) | Q4_K_S | 4.6 | fast, recommended |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q4_K_M.gguf) | Q4_K_M | 4.8 | fast, recommended |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q5_K_S.gguf) | Q5_K_S | 5.4 |  |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q5_K_M.gguf) | Q5_K_M | 5.5 |  |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q6_K.gguf) | Q6_K | 6.4 | very good quality |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.Q8_0.gguf) | Q8_0 | 8.2 | fast, best quality |\n| [GGUF](https://huggingface.co/mradermacher/ReSearch-Qwen-7B-GGUF/resolve/main/ReSearch-Qwen-7B.f16.gguf) | f16 | 15.3 | 16 bpw, overkill |\n\nHere is a handy graph by ikawrakow comparing some lower-quality quant\ntypes (lower is better):\n\n![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)\n\n",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "qwen2",
    "text-generation",
    "function-calling",
    "tool-calling",
    "codex",
    "local-llm",
    "6gb-vram",
    "ollama",
    "code-assistant",
    "api-tools",
    "openai-alternative",
    "conversational",
    "en",
    "dataset:RUC-NLPIR/FlashRAG_datasets",
    "base_model:agentrl/ReSearch-Qwen-7B",
    "base_model:quantized:agentrl/ReSearch-Qwen-7B",
    "license:mit",
    "endpoints_compatible",
    "region:us"
  ],
  "likes": 2,
  "downloads": 228,
  "gated": false,
  "private": false,
  "last_modified": "2025-09-21T20:04:37.000Z",
  "created_at": "2025-09-21T19:53:23.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "transformers"
}

{
  "_id": "68d057b36a7a54e7e83ca7fd",
  "id": "Manojb/Qwen-7B-toolcalling-ReSearch-gguf-Q8_0-codex",
  "modelId": "Manojb/Qwen-7B-toolcalling-ReSearch-gguf-Q8_0-codex",
  "sha": "fcd75b8a6b50ec1e6b87e184100dc19cea77b66e",
  "createdAt": "2025-09-21T19:53:23.000Z",
  "lastModified": "2025-09-21T20:04:37.000Z",
  "author": "Manojb",
  "downloads": 228,
  "likes": 2,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "transformers",
  "siblings_count": 14
}