benhaotang/rombos-coder-v2.5-qwen-7b-gguf_cline overview

{
  "metadata": {},
  "card_data": {
    "base_model": "rombodawg/Rombos-Coder-V2.5-Qwen-7b",
    "tags": [
      "llama-cpp",
      "gguf-my-repo",
      "cline"
    ],
    "frontmatter": {
      "base_model": "rombodawg/Rombos-Coder-V2.5-Qwen-7b",
      "tags": [
        "llama-cpp",
        "gguf-my-repo",
        "cline"
      ]
    },
    "hero_image_url": "",
    "summary": "This model was converted to GGUF format from rombodawg/Rombos-Coder-V2.5-Qwen-7b using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nbase_model: rombodawg/Rombos-Coder-V2.5-Qwen-7b\ntags:\n- llama-cpp\n- gguf-my-repo\n- cline\n---\n\n# benhaotang/Rombos-Coder-V2.5-Qwen-7b-Q8_0-GGUF\nThis model was converted to GGUF format from [`rombodawg/Rombos-Coder-V2.5-Qwen-7b`](https://huggingface.co/rombodawg/Rombos-Coder-V2.5-Qwen-7b) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.\nRefer to the [original model card](https://huggingface.co/rombodawg/Rombos-Coder-V2.5-Qwen-7b) for more details on the model.\n\n## Use with Cline and Ollama\n\nuse this template file from https://github.com/maryasov/ollama-models-instruct-for-cline\n\n```\nFROM rombos-coder-v2.5-qwen-7b-q8_0.gguf\nTEMPLATE \"\"\"{{- /* Initial system message with core instructions */ -}}\n{{- if .Messages }}\n{{- if or .System .Tools }}\n<|im_start|>system\n{{- if .System }}\n{{ .System }}\n{{- end }}                                                                                                                                                                                                  {{- if .Tools }}\n# Tools and XML Schema\nYou have access to the following tools. Each tool must be used according to this XML schema:\n\n<tools>\n{{- range .Tools }}\n{{ .Function }}\n{{- end }}\n</tools>\n\n## Tool Use Format\n1. Think about the approach in <thinking> tags\n2. Call tool using XML format:\n   <tool_name>\n     <param_name>value</param_name>\n   </tool_name>\n3. Process tool response from:\n   <tool_response>result</tool_response>\n{{- end }}\n<|im_end|>\n{{- end }}\n\n{{- /* Message handling loop */ -}}\n{{- range $i, $_ := .Messages }}\n{{- $last := eq (len (slice $.Messages $i)) 1 }}\n\n{{- /* User messages */ -}}\n{{- if eq .Role \"user\" }}\n<|im_start|>user\n{{ .Content }}\n<|im_end|>\n\n{{- /* Assistant messages */ -}}\n{{- else if eq .Role \"assistant\" }}\n<|im_start|>assistant\n{{- if .Content }}\n{{ .Content }}\n{{- else if .ToolCalls }}\n{{- range .ToolCalls }}\n<thinking>\n[Analysis of current state and next steps]\n</thinking>\n\n<{{ .Function.Name }}>\n{{- range $key, $value := .Function.Arguments }}\n<{{ $key }}>{{ $value }}</{{ $key }}>\n{{- end }}\n</{{ .Function.Name }}>\n{{- end }}\n{{- end }}\n{{- if not $last }}<|im_end|>{{- end }}\n\n{{- /* Tool response handling */ -}}\n{{- else if eq .Role \"tool\" }}\n<|im_start|>user\n<tool_response>\n{{ .Content }}\n</tool_response>\n<|im_end|>\n{{- end }}\n\n{{- /* Prepare for next assistant response if needed */ -}}\n{{- if and (ne .Role \"assistant\") $last }}\n<|im_start|>assistant\n{{- end }}\n{{- end }}\n\n{{- /* Handle single message case */ -}}\n{{- else }}\n{{- if .System }}\n<|im_start|>system\n{{ .System }}\n<|im_end|>\n{{- end }}\n\n{{- if .Prompt }}\n<|im_start|>user\n{{ .Prompt }}\n<|im_end|>\n{{- end }}\n\n<|im_start|>assistant\n{{- end }}\n{{ .Response }}\n{{- if .Response }}<|im_end|>{{- end }}\n\"\"\"\nPARAMETER repeat_last_n 64\nPARAMETER repeat_penalty 1.1\nPARAMETER stop \"<|im_start|>\"\nPARAMETER stop \"<|im_end|>\"\nPARAMETER stop \"<|endoftext|>\"\nPARAMETER temperature 0.1\nPARAMETER top_k 40\nPARAMETER top_p 0.9\n```\n\n## Use with llama.cpp\nInstall llama.cpp through brew (works on Mac and Linux)\n\n```bash\nbrew install llama.cpp\n\n```\nInvoke the llama.cpp server or the CLI.\n\n### CLI:\n```bash\nllama-cli --hf-repo benhaotang/Rombos-Coder-V2.5-Qwen-7b-Q8_0-GGUF --hf-file rombos-coder-v2.5-qwen-7b-q8_0.gguf -p \"The meaning to life and the universe is\"\n```\n\n### Server:\n```bash\nllama-server --hf-repo benhaotang/Rombos-Coder-V2.5-Qwen-7b-Q8_0-GGUF --hf-file rombos-coder-v2.5-qwen-7b-q8_0.gguf -c 2048\n```\n\nNote: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.\n\nStep 1: Clone llama.cpp from GitHub.\n```\ngit clone https://github.com/ggerganov/llama.cpp\n```\n\nStep 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).\n```\ncd llama.cpp && LLAMA_CURL=1 make\n```\n\nStep 3: Run inference through the main binary.\n```\n./llama-cli --hf-repo benhaotang/Rombos-Coder-V2.5-Qwen-7b-Q8_0-GGUF --hf-file rombos-coder-v2.5-qwen-7b-q8_0.gguf -p \"The meaning to life and the universe is\"\n```\nor \n```\n./llama-server --hf-repo benhaotang/Rombos-Coder-V2.5-Qwen-7b-Q8_0-GGUF --hf-file rombos-coder-v2.5-qwen-7b-q8_0.gguf -c 2048\n```",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "llama-cpp",
    "gguf-my-repo",
    "cline",
    "base_model:rombodawg/Rombos-Coder-V2.5-Qwen-7b",
    "base_model:quantized:rombodawg/Rombos-Coder-V2.5-Qwen-7b",
    "endpoints_compatible",
    "region:us",
    "imatrix",
    "conversational"
  ],
  "likes": 10,
  "downloads": 82,
  "gated": false,
  "private": false,
  "last_modified": "2024-11-15T23:35:36.000Z",
  "created_at": "2024-11-05T21:56:44.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "672a949cea85c25f4292191c",
  "id": "benhaotang/Rombos-Coder-V2.5-Qwen-7b-GGUF_cline",
  "modelId": "benhaotang/Rombos-Coder-V2.5-Qwen-7b-GGUF_cline",
  "sha": "17f94847e7054637b2532d92ab07cf845a1297ea",
  "createdAt": "2024-11-05T21:56:44.000Z",
  "lastModified": "2024-11-15T23:35:36.000Z",
  "author": "benhaotang",
  "downloads": 82,
  "likes": 10,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 4
}