quantfactory/llama-3.2-taiwan-legal-3b-instruct-gguf overview

{
  "metadata": {},
  "card_data": {
    "library_name": "transformers",
    "license": "llama3.2",
    "base_model": "meta-llama/Llama-3.2-3B-Instruct",
    "datasets": [
      "lianghsun/tw-emergency-medicine-bench",
      "lianghsun/tw-legal-nlp",
      "lianghsun/tw-legal-synthetic-qa",
      "lianghsun/tw-law-article-qa",
      "lianghsun/tw-judgment-qa",
      "lianghsun/tw-bar-examination-2020-chat"
    ],
    "tags": [
      "legal",
      "TW",
      "Taiwan",
      "ROC",
      "llama-factory",
      "full",
      "generated_from_trainer"
    ],
    "model-index": [
      {
        "name": "train_2024-10-17",
        "results": []
      }
    ],
    "new_version": "lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct",
    "language": [
      "zh"
    ],
    "pipeline_tag": "text-generation",
    "frontmatter": {},
    "hero_image_url": "https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ",
    "summary": "This is quantized version of lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct created using llama.cpp # Original Model Card  # Model Card for Model lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct !image/png 基於 meta-llama/Llama-3.2-3B-Instruct 模型，透過中華民國台灣法律條文及判決書等相關資料集進行微調。",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "\n---\n\nlibrary_name: transformers\nlicense: llama3.2\nbase_model: meta-llama/Llama-3.2-3B-Instruct\ndatasets:\n- lianghsun/tw-emergency-medicine-bench\n- lianghsun/tw-legal-nlp\n- lianghsun/tw-legal-synthetic-qa\n- lianghsun/tw-law-article-qa\n- lianghsun/tw-judgment-qa\n- lianghsun/tw-bar-examination-2020-chat\ntags:\n- legal\n- TW\n- Taiwan\n- ROC\n- llama-factory\n- full\n- generated_from_trainer\nmodel-index:\n- name: train_2024-10-17\n  results: []\nnew_version: lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct\nlanguage:\n- zh\npipeline_tag: text-generation\n\n---\n\n[![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory)\n\n\n# QuantFactory/Llama-3.2-Taiwan-Legal-3B-Instruct-GGUF\nThis is quantized version of [lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct](https://huggingface.co/lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct) created using llama.cpp\n\n# Original Model Card\n\n\n<!-- This model card has been generated automatically according to the information the Trainer had access to. You\nshould probably proofread and complete it, then remove this comment. -->\n\n# Model Card for Model lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct\n\n![image/png](https://cdn-uploads.huggingface.co/production/uploads/618dc56cbc345ca7bf95f3cd/W6-UDg0_cNm4WJVlR9tiD.png)\n\n基於 [meta-llama/Llama-3.2-3B-Instruct](meta-llama/Llama-3.2-3B-Instruct) 模型，透過中華民國台灣法律條文及判決書等相關資料集進行微調。\n\n## Model Update History\n\n| Update Date  | Model Version         | Key Changes                         |\n|--------------|-----------------------|-------------------------------------|\n| 2024-10-17   | v1.1.0                | Experimental fine-tuning on v1.0.0 with added legal code data from the Republic of China (Taiwan) |\n| 2024-10-10   | v1.0.0                | Full model training completed, but missing legal code data for the Republic of China (Taiwan) |\n| 2024-09-27   | v0.1.0                | Model v0.1.0 released, but training was interrupted after 3 epochs due to lack of compute resources |\n\n\n## Model Details\n\n### Model Description\n\n基於 [meta-llama/Llama-3.2-3B-Instruct](meta-llama/Llama-3.2-3B-Instruct) 模型，此微調過程使用了來自中華民國台灣的法律條文與相關判決書資料集，以提升模型在法律領域的專業知識與應用能力。這些資料集涵蓋了法律條文的結構、判決書的格式，法庭上常見的法律語言與術語，並包含了部分法律資料科學任務的應用，使模型能夠更準確地理解和處理與台灣法律體系相關的問題。經過這些微調，模型將能夠更好地為法律專業人士提供幫助，並在台灣法制框架內提供更精準的回應與建議。\n\n- **Developed by:** [Huang Liang Hsun](https://www.linkedin.com/in/lianghsunhuang)\n- **Model type:** LlamaForCausalLM\n- **Language(s) (NLP)**: 主要處理繁體中文（zh-tw），針對中華民國台灣的法律用語與判決書進行微調。\n- **License**: [llama3.2](https://huggingface.co/meta-llama/Llama-3.2-1B/blob/main/LICENSE.txt)\n- **Finetuned from model**: [meta-llama/Llama-3.2-3B-Instruct](meta-llama/Llama-3.2-3B-Instruct)\n\n### Model Sources\n\n- **Repository:** [lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct](https://huggingface.co/lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct)\n- **Demo:** (WIP)\n\n## Uses\n\n<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->\n\n### Direct Use\n\n<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->\n\n此模型可以直接用於理解和生成繁體中文法律文本，適合需要處理台灣法律相關問題的應用場景。模型預設的指令和回應能夠有效提供法律資訊、釐清法律條文、並生成符合法律專業的回應。其直接使用範圍包括但不限於法律資訊查詢、法律文本摘要、和基本的法條對話。\n\n### Downstream Use\n\n<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->\n\n經過微調後，該模型可用於更具體的法律任務，如自動判決書分析、法律實體識別（NER）、法規編號轉換，以及法律合規審查輔助。此模型可以無縫集成至法律數據科學應用或法律技術（LegalTech）系統中，幫助法律專業人士或企業提升工作效率。\n\n### Out-of-Scope Use\n\n<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->\n\n該模型並不適用於非法律相關領域的生成任務，且不應用於進行可能涉及誤導或錯誤的法律建議，尤其是在未經專業審查的情況下。避免將模型用於未經授權或非法用途，如生成具爭議性或具偏見的法律建議。\n\n## Bias, Risks, and Limitations\n\n模型在生成法律條文和判決書內容時，可能會生成虛構或不存在的法條或判決書內容，這是模型的內在限制之一。使用者在參考這些資料時，應謹慎檢查生成的內容，並避免將模型輸出視為法律依據。建議在實際應用中，將模型生成的結果與可靠的法律見解和來源進行比對，確保準確性、合法性和適用性。\n\n### Recommendations\n\n<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->\n\n此模型雖然經過法律文本的微調，但在於法律文本的數量及基礎模型為 SLM，模型能力仍有極限，使用者應注意以下風險與限制：\n\n1. **偏見風險**：\n模型可能會反映其訓練資料中的潛在偏見。由於法律文本的特定性，模型可能更熟悉某些法規、條文或判決案例，而在其他領域表現較弱。特別是在處理不常見的法律問題或未被訓練過的新法規時，模型的輸出可能會帶有偏見。\n\n2. **技術限制**：\n雖然模型能夠處理大部分的法律文本，但對於結構極其複雜或語言模棱兩可的法律條文，模型可能無法產生精確的回答。使用者應避免完全依賴模型的輸出，尤其在法律決策過程中，建議進行額外的專業檢查。\n\n3. **法律責任**：\n模型並非專業法律顧問，因此其生成的回應不應被視為正確的法律建議。使用者應確保在理性且專業背景下進行模型的應用，並避免在關鍵決策中過度依賴模型。\n\n4. **誤用風險**：\n不當使用模型進行錯誤或誤導性的法律建議，可能對個人或企業造成負面影響。使用者應謹慎應用模型於合規或法律相關任務中，並保持對其輸出的檢視及校正。\n\n為了減少這些風險，建議使用者在應用模型輸出時進行二次檢查，特別是在涉及法律決策的情境中。本模型現階段為提供法律科技領域進行大語言模型研究，並非取代專業法律工作者之專業建議。\n\n## How to Get Started with the Model\n\n<!-- Use the code below to get started with the model. -->\n\n### Using vLLM\n\n要使用 [vLLM Docker image](https://docs.vllm.ai/en/latest/serving/deploying_with_docker.html) 來啟動此模型，您可以按照以下操作：\n```\ndocker run --runtime nvidia --gpus all \\\n    -v ~/.cache/huggingface:/root/.cache/huggingface \\\n    --env \"HUGGING_FACE_HUB_TOKEN=<secret>\" \\\n    -p 8000:8000 \\\n    --ipc=host \\\n    vllm/vllm-openai:latest \\\n    --model lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct\n```\n## Training Details\n\n### Training Data (for v1.1.0)\n\n- [lianghsun/tw-legal-nlp](https://huggingface.co/datasets/lianghsun/tw-legal-nlp)\n- [lianghsun/tw-legal-synthetic-qa](https://huggingface.co/datasets/lianghsun/tw-legal-synthetic-qa)\n- [lianghsun/tw-law-article-qa](https://huggingface.co/datasets/lianghsun/tw-law-article-qa)\n- [lianghsun/tw-judgment-qa](https://huggingface.co/datasets/lianghsun/tw-judgment-qa)\n- [lianghsun/tw-bar-examination-2020-chat](https://huggingface.co/datasets/lianghsun/tw-bar-examination-2020-chat)\n- [lianghsun/tw-emergency-medicine-bench](https://huggingface.co/datasets/lianghsun/tw-emergency-medicine-bench)\n\n### Training procedure\n\n#### Preprocessing\n\n無。基本上我們並沒有針對 [meta-llama/Llama-3.2-3B-Instruct](meta-llama/Llama-3.2-3B-Instruct) 做任何的預訓練或更改其模型架構；Tokenizer 也是採用原生所提供的。\n\n#### Training hyperparameters (for v1.1.0)\n\nThe following hyperparameters were used during training:\n\n- **learning_rate:** 0.0004378 (value at epoch 3.9)\n- **train_batch_size:** 12\n- **eval_batch_size:** Not specified\n- **seed:** Not specified\n- **distributed_type:** single-GPU\n- **num_devices:** 1\n- **gradient_accumulation_steps:** 512\n- **total_train_batch_size:** 6144 (train_batch_size * gradient_accumulation_steps)\n- **optimizer:** AdamW\n- **lr_scheduler_type:** cosine\n- **lr_scheduler_warmup_steps:** 100\n- **num_epochs:** 15\n- **grad_norm:** 0.0899 (value at epoch 3.9)\n- **global_step:** 645\n\n### Speeds, Sizes, Times (for v1.1.0)\n\n- **Duration**: 92h 27m 40s\n- **Train runtime**: 92h 27m 40s\n- **Train samples per second**: Not directly available\n- **Train steps per second**: Approximately 0.002 steps/s\n- **Total training FLOPs**: Not directly provided\n- **Train loss**: 0.0512 (at epoch 3.9)\n\n## Evaluation\n\n<!-- This section describes the evaluation protocols and provides the results. -->\n\n### Testing Data, Factors & Metrics\n\n#### Testing Data\n\n<!-- This should link to a Dataset Card if possible. -->\n\n**Note**: ..(WIP)..\n\n#### Factors\n\n<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->\n\n**Note**: ..(WIP)..\n\n#### Metrics\n\n<!-- These are the evaluation metrics being used, ideally with a description of why. -->\n\n**Note**: ..(WIP)..\n\n### Results\n\n**Note**: ..(WIP)..\n\n#### Summary\n\n**Note**: ..(WIP)..\n\n## Model Examination\n\n<!-- Relevant interpretability work for the model goes here -->\n\n### 法條回覆\n\n**Note**: ..(WIP)..\n\n### 判決書內容\n\n**Note**: ..(WIP)..\n\n### 法律 NLP 任務\n\n**Note**: ..(WIP)..\n\n## Environmental Impact (for v1.1.0)\n\n- **Hardware Type:** 1 x NVIDIA H100 NVL 80GB\n- **Hours used:** 92h 27m 40s\n- **Cloud Provider:** N/A\n- **Compute Region:** N/A\n- **Carbon Emitted:** N/A                        \n\n## Technical Specifications\n\n### Model Architecture and Objective\n\n本模型基於 `meta-llama/Llama-3.2-3B-Instruct`，使用自回歸 Transformer 架構進行語言建模。該模型的主要目標是提升對台灣法律文本的理解與生成能力，尤其是針對判決書、法條的專業處理與應用。透過專門設計的法律文本集進行微調，模型能更精確地回答法律問題並提供相關建議。\n\n### Compute Infrastructure\n\n#### Hardware (for v1.1.0)\n\n- 1 x NVIDIA H100 NVL 80GB\n\n#### Software\n\n- 微調過程使用了 [hiyouga/LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) 框架進行訓練。\n\n## Citation\n\n無。\n\n## Glossary\n\n<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->\n無。\n\n## More Information\n\n### 算力\n儘管我們已準備了許多關於中華民國台灣法律領域的資料集，但由於算力資源有限，**無法將所有資料集進行完整訓練**（是的，我們並沒有將全部資料集都進行訓練，僅取出被認為最基礎的法律文本），導致模型尚未達到最佳表現。因此，目前的 checkpoint 是基於有限資源的版本。如果您有意願贊助算力，歡迎與我聯繫。我相信，若能將更多已準備但尚未納入訓練的法律語料進行微調，該模型將能達到繁體中文法律領域的最佳表現。\n\n### 持績更新\n此模型如有進一步資源，將會不定期更新。\n\n## Model Card Authors\n\n[Huang Liang Hsun](https://www.linkedin.com/in/lianghsunhuang)\n\n## Model Card Contact\n\n[Huang Liang Hsun](https://www.linkedin.com/in/lianghsunhuang)\n\n### Framework versions\n\n- Transformers 4.45.2\n- Pytorch 2.4.1+cu121\n- Datasets 2.21.0\n- Tokenizers 0.20.0\n",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "legal",
    "TW",
    "Taiwan",
    "ROC",
    "llama-factory",
    "full",
    "generated_from_trainer",
    "text-generation",
    "zh",
    "dataset:lianghsun/tw-emergency-medicine-bench",
    "dataset:lianghsun/tw-legal-nlp",
    "dataset:lianghsun/tw-legal-synthetic-qa",
    "dataset:lianghsun/tw-law-article-qa",
    "dataset:lianghsun/tw-judgment-qa",
    "dataset:lianghsun/tw-bar-examination-2020-chat",
    "base_model:meta-llama/Llama-3.2-3B-Instruct",
    "base_model:quantized:meta-llama/Llama-3.2-3B-Instruct",
    "license:llama3.2",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 11,
  "downloads": 340,
  "gated": false,
  "private": false,
  "last_modified": "2024-11-02T14:47:13.000Z",
  "created_at": "2024-10-03T09:15:50.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "transformers"
}

{
  "_id": "66fe60c65f2dc96cdb55dd4e",
  "id": "QuantFactory/Llama-3.2-Taiwan-Legal-3B-Instruct-GGUF",
  "modelId": "QuantFactory/Llama-3.2-Taiwan-Legal-3B-Instruct-GGUF",
  "sha": "25947ba3ba6fc1a2e9c977dfa2ca1b6abcdb1756",
  "createdAt": "2024-10-03T09:15:50.000Z",
  "lastModified": "2024-11-02T14:47:13.000Z",
  "author": "QuantFactory",
  "downloads": 340,
  "likes": 11,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "transformers",
  "siblings_count": 16
}