richarderkhov/lubocido_-_ko-llama3-luxia-8b-it-gguf overview

{
  "metadata": {},
  "card_data": {
    "frontmatter": {},
    "hero_image_url": "",
    "summary": "Quantization made by Richard Erkhov. Github Discord Request more models Ko-Llama3-Luxia-8B-it - GGUF | Name | Quant method | Size | | ---- | ---- | ---- | | Ko-Llama3-Luxia-8B-it.Q2_K.gguf | Q2_K | 3.04GB | | Ko-Llama3-Luxia-8B-it.IQ3_XS.gguf | IQ3_XS | 3.36GB | | Ko-Llama3-Luxia-8B-it.IQ3_S.gguf | IQ3_S | 3.51GB | | Ko-Llama3-Luxia-8B-it.Q3_K_S.gguf | Q3_K_S | 3.5GB | | Ko-Llama3-Luxia-8B-it.IQ3_M.gguf | IQ3_M | 3.61GB | | Ko-Llama3-Luxia-8B-it.Q3_K.gguf | Q3_K | 3.83GB | | Ko-Llama3-Luxia-8B-it.Q3_K_M.gguf | Q3_K_M | 3.83GB | | Ko-Llama3-Luxia-8B-it.Q3_K_L.gguf | Q3_K_L | 4.11GB | | Ko-Llama3-Luxia-8B-it.IQ4_XS.gguf | IQ4_XS | 4.27GB | | Ko-Llama3-Luxia-8B-it.Q4_0.gguf | Q4_0 | 4.43GB | | Ko-Llama3-Luxia-8B-it.IQ4_NL.gguf | IQ4_NL | 4.48GB | | Ko-Llama3-Luxia-8B-it.Q4_K_S.gguf | Q4_K_S | 4.46GB | | Ko-Llama3-Luxia-8B-it.Q4_K.gguf | Q4_K | 4.68GB | | Ko-Llama3-Luxia-8B-it.Q4_K_M.gguf | Q4_K_M | 4.68GB | | Ko-Llama3-Luxia-8B-it.Q4_1.gguf | Q4_1 | 4.88GB | | Ko-Llama3-Luxia-8B-it.Q5_0.gguf | Q5_0 | 5.32GB | | Ko-Llama3-Luxia-8B-it.Q5_K_S.gguf | Q5_K_S | 5.32GB | | Ko-Llama3-Luxia-8B-it.Q5_K.gguf | Q5_K | 5.44GB | | Ko-Llama3-Luxia-8B-it.Q5_K_M.gguf | Q5_K_M | 5.44GB | | Ko-Llama3-Luxia-8B-it.Q5_1.gguf | Q5_1 | 5.76GB | | Ko-Llama3-Luxia-8B-it.Q6_K.gguf | Q6_K | 6.25GB | | Ko-Llama3-Luxia-8B-it.Q8_0.gguf | Q8_0 | 8.1GB | Original model description: --- license: llama3 language: base_model: saltlux/Ko-Llama3-Luxia-8B ---",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "Quantization made by Richard Erkhov.\n\n[Github](https://github.com/RichardErkhov)\n\n[Discord](https://discord.gg/pvy7H8DZMG)\n\n[Request more models](https://github.com/RichardErkhov/quant_request)\n\n\nKo-Llama3-Luxia-8B-it - GGUF\n- Model creator: https://huggingface.co/lubocido/\n- Original model: https://huggingface.co/lubocido/Ko-Llama3-Luxia-8B-it/\n\n\n| Name | Quant method | Size |\n| ---- | ---- | ---- |\n| [Ko-Llama3-Luxia-8B-it.Q2_K.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q2_K.gguf) | Q2_K | 3.04GB |\n| [Ko-Llama3-Luxia-8B-it.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.IQ3_XS.gguf) | IQ3_XS | 3.36GB |\n| [Ko-Llama3-Luxia-8B-it.IQ3_S.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.IQ3_S.gguf) | IQ3_S | 3.51GB |\n| [Ko-Llama3-Luxia-8B-it.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q3_K_S.gguf) | Q3_K_S | 3.5GB |\n| [Ko-Llama3-Luxia-8B-it.IQ3_M.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.IQ3_M.gguf) | IQ3_M | 3.61GB |\n| [Ko-Llama3-Luxia-8B-it.Q3_K.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q3_K.gguf) | Q3_K | 3.83GB |\n| [Ko-Llama3-Luxia-8B-it.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q3_K_M.gguf) | Q3_K_M | 3.83GB |\n| [Ko-Llama3-Luxia-8B-it.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q3_K_L.gguf) | Q3_K_L | 4.11GB |\n| [Ko-Llama3-Luxia-8B-it.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.IQ4_XS.gguf) | IQ4_XS | 4.27GB |\n| [Ko-Llama3-Luxia-8B-it.Q4_0.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q4_0.gguf) | Q4_0 | 4.43GB |\n| [Ko-Llama3-Luxia-8B-it.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.IQ4_NL.gguf) | IQ4_NL | 4.48GB |\n| [Ko-Llama3-Luxia-8B-it.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q4_K_S.gguf) | Q4_K_S | 4.46GB |\n| [Ko-Llama3-Luxia-8B-it.Q4_K.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q4_K.gguf) | Q4_K | 4.68GB |\n| [Ko-Llama3-Luxia-8B-it.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q4_K_M.gguf) | Q4_K_M | 4.68GB |\n| [Ko-Llama3-Luxia-8B-it.Q4_1.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q4_1.gguf) | Q4_1 | 4.88GB |\n| [Ko-Llama3-Luxia-8B-it.Q5_0.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q5_0.gguf) | Q5_0 | 5.32GB |\n| [Ko-Llama3-Luxia-8B-it.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q5_K_S.gguf) | Q5_K_S | 5.32GB |\n| [Ko-Llama3-Luxia-8B-it.Q5_K.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q5_K.gguf) | Q5_K | 5.44GB |\n| [Ko-Llama3-Luxia-8B-it.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q5_K_M.gguf) | Q5_K_M | 5.44GB |\n| [Ko-Llama3-Luxia-8B-it.Q5_1.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q5_1.gguf) | Q5_1 | 5.76GB |\n| [Ko-Llama3-Luxia-8B-it.Q6_K.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q6_K.gguf) | Q6_K | 6.25GB |\n| [Ko-Llama3-Luxia-8B-it.Q8_0.gguf](https://huggingface.co/RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf/blob/main/Ko-Llama3-Luxia-8B-it.Q8_0.gguf) | Q8_0 | 8.1GB |\n\n\n\n\nOriginal model description:\n---\nlicense: llama3\nlanguage:\n- ko\nbase_model: saltlux/Ko-Llama3-Luxia-8B\n---\n## Model Details\n\nSaltlux, AI Labs 에서 개발한 [saltlux/Ko-Llama3-Luxia-8B](https://huggingface.co/saltlux/Ko-Llama3-Luxia-8B) 모델을 Instruction Fine tuning한 모델입니다.  \n사용된 데이터셋으로 [maywell/ko_wikidata_QA](https://huggingface.co/datasets/maywell/ko_wikidata_QA)를 사용하였으며 SFTTrainer를 통해 3ep로 학습했습니다.  \ninstruction prompt는 Qwen2 모델과 동일하게 적용시켰습니다.\n\n```python\n<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWhat is the Qwen2?<|im_end|>\n<|im_start|>assistant\nQwen2 is the new series of Qwen large language models<|im_end|>\n<|im_start|>user\nTell me more<|im_end|>\n<|im_start|>assistant\n```\n\n## HyperParameter\n- num_train_epochs = 3\n- warmup_steps=0.03\n- learning_rate=1e-5\n- optim=\"adamw_torch_fused\"\n\n## Evaluation with Langchain\napply_chat_tempalte이 적용되어있지 않아 랭체인에서 프롬프트로 직접 입력하여 평가해 볼 수 있습니다.  \n\n```python\nmodel_id = \"lubocido/Ko-Llama3-Luxia-8B-it\"\ndevice = \"cuda:0\"\n\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = AutoModelForCausalLM.from_pretrained(model_id,, device_map = device, torch_dtype = torch.bfloat16)\n\ntokenizer.padding_side = 'right'\ntokenizer.pad_token = tokenizer.eos_token\n\nsys_message = \"\"\"당신은 친절한 챗봇으로서 상대방의 요청에 최대한 자세하고 친절하게 답해야합니다. \n사용자가 제공하는 정보를 세심하게 분석하여 사용자의 의도를 신속하게 파악하고 그에 따라 답변을 생성해야합니다.\n항상 매우 자연스러운 한국어로 응답하세요.\"\"\"\n\nquestion = \"리눅스에서 프로세스를 죽이는 명령어가 뭐지?\"\n\ntemplate = \"\"\"\n<|im_start|>system\\n{sys_message}<|im_end|>\n<|im_start|>user\\n{question}<|im_end|>\n<|im_start|>assistant\n\"\"\"\n\ninput_data = {\n    'sys_message' : sys_message,\n    'question' : question,\n}\n\nprompt = PromptTemplate(template=template, input_variables=['sys_message', 'question'])\n\npipe = pipeline('text-generation', model=model, tokenizer=tokenizer, device_map=device, do_sample = True, max_length = 512, temperature = 0.1, repetition_penalty=1.2, num_beams=1,top_k=20,top_p=0.9)\n\nlangchain_pipeline = HuggingFacePipeline(pipeline=pipe)\n\nchains = LLMChain(llm=langchain_pipeline, prompt=prompt, output_parser=StrOutputParser(), verbose=True)\n\nprint(chains.invoke(input=input_data)['text'])\n```\n\n```\n<|im_start|>user\n리눅스에서 프로세스를 죽이는 명령어가 뭐지?<|im_end|>\n<|im_start|>assistant\n프로세스는 운영 체제가 실행 중인 프로그램으로, 프로세스 ID(PID)라는 고유한 식별자를 가지고 있습니다.\n프로세스가 종료되면 시스템 자원이 해제됩니다.   리눅스의 경우 kill 명령어를 통해 프로세스를 종료할 수 있으며, 이 명령어는 PID 또는 이름과 같은 다양한 방법으로 프로세스를 찾아서 종료시킬 수 있습니다.\n또한 SIGKILL 신호를 보내거나 -9 옵션을 사용하면 강제적으로 프로세스를 종료할 수도 있습니다.\n그러나 일부 프로세스는 강제 종료될 때 문제를 일으킬 수 있으므로 주의해서 사용해야 합니다.<|im_end|>\n```\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 276,
  "gated": false,
  "private": false,
  "last_modified": "2024-10-05T05:45:02.000Z",
  "created_at": "2024-10-04T23:00:07.000Z",
  "pipeline_tag": "",
  "library_name": ""
}

{
  "_id": "6700737700dc08cbf2487501",
  "id": "RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf",
  "modelId": "RichardErkhov/lubocido_-_Ko-Llama3-Luxia-8B-it-gguf",
  "sha": "1f84a8532bca5748bdc60edcb720e26d106bd1d6",
  "createdAt": "2024-10-04T23:00:07.000Z",
  "lastModified": "2024-10-05T05:45:02.000Z",
  "author": "RichardErkhov",
  "downloads": 276,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "",
  "library_name": "",
  "siblings_count": 24
}