---

license: llama3

base_model: Jeesup/svd-safety-l3_swift_remove50_swapgapiter_evfront_b010

tags:

- gguf

- llama

- llama3

- svd

- compression

- safety

- interpretability

- quantized

language:

- en

pipeline_tag: text-generation

library_name: gguf

---

svd-safety-l3_swift_remove50_swapgapiter_evfront_b010 — GGUF

GGUF quantizations of Jeesup/svd-safety-l3_swift_remove50_swapgapiter_evfront_b010.

About the original model

A Llama-3-8B-Instruct checkpoint compressed with Swift-SVD with LoRA recovery to 50.0% of dense parameters, then edited by 10 of 10 rounds of iterative parameter-neutral swap selected by the gap_iter rule (up to 0.1% of dense parameters per round; 1.0% total budget).

This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is not a general-purpose chat model. Some arms of the original study are deliberately safety-degraded relative to Llama-3-8B-Instruct. Treat this model as an experimental subject and evaluate it yourself before drawing conclusions.

Key features (original model)

• Base: meta-llama/Meta-Llama-3-8B-Instruct
• Compression: Swift-SVD + LoRA recovery (49.97% of parameters removed)
• Selection rule: gap_iter, restore budget 1.0% of dense parameters
• Eviction: frontier (layers 0-1 frozen, at most 5% of a matrix's rank)
• Swap value: insert
• Seed: 42

Reported metrics (original model)

| Metric | Value |

|---|---|

| AdvBench ASR (HarmBench judge) | 0.0942 |

| StrongREJECT ASR (HarmBench judge) | 0.2556 |

| Macro over-refusal (WildGuard) | 0.1118 |

| WikiText-2 perplexity | 55.4647 |

Available quantizations

| File | Bits | Approx. size | Use case |

|---|---|---|---|

| model_f16.gguf | 16 | ~16.1 GB | Full precision reference, source for further quantization |

| model_q8_0.gguf | 8 | ~8.5 GB | Near-lossless |

| model_q6_k.gguf | 6 | ~6.6 GB | Very high quality |

| model_q5_k_m.gguf | 5 | ~5.7 GB | High quality, recommended balance |

| model_q5_k_s.gguf | 5 | ~5.6 GB | High quality, slightly smaller |

| model_q4_k_m.gguf | 4 | ~4.9 GB | Best general-purpose size/quality trade-off |

| model_q4_k_s.gguf | 4 | ~4.7 GB | Smaller 4-bit variant |

| model_q3_k_l.gguf | 3 | ~4.3 GB | Low memory, noticeable quality loss |

| model_q3_k_m.gguf | 3 | ~4.0 GB | Low memory |

| model_q3_k_s.gguf | 3 | ~3.7 GB | Very low memory |

| model_q2_k.gguf | 2 | ~3.2 GB | Smallest, significant quality loss |

Memory requirements (approximate)

| Quant | RAM / VRAM (4K context) |

|---|---|

| F16 | ~18 GB |

| Q8_0 | ~10.5 GB |

| Q6_K | ~8.5 GB |

| Q5_K_M / Q5_K_S | ~7.5 GB |

| Q4_K_M / Q4_K_S | ~6.5-7 GB |

| Q3_K_L / Q3_K_M / Q3_K_S | ~5.5-6.5 GB |

| Q2_K | ~5 GB |

Usage

llama.cpp

huggingface-cli download tinyopsec/svd-safety-l3-swift-remove50-swapgapiter-evfront-b010-GGUF \
  model_q4_k_m.gguf --local-dir .

./llama-cli -m model_q4_k_m.gguf -p "Who are you?" -n 256

llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="tinyopsec/svd-safety-l3-swift-remove50-swapgapiter-evfront-b010-GGUF",
    filename="model_q4_k_m.gguf",
    n_ctx=4096,
)

out = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Who are you?"}]
)
print(out["choices"][0]["message"]["content"])

LM Studio

1. Open LM Studio and go to the Search tab.
2. Search for tinyopsec/svd-safety-l3-swift-remove50-swapgapiter-evfront-b010-GGUF.
3. Select a quantization (e.g. Q4_K_M) and download.
4. Load the model in the Chat tab.

Ollama

huggingface-cli download tinyopsec/svd-safety-l3-swift-remove50-swapgapiter-evfront-b010-GGUF \
  model_q4_k_m.gguf --local-dir .

cat > Modelfile <<EOF
FROM ./model_q4_k_m.gguf
EOF

ollama create svd-safety-l3 -f Modelfile
ollama run svd-safety-l3

Original model

• Jeesup/svd-safety-l3_swift_remove50_swapgapiter_evfront_b010
• Base: meta-llama/Meta-Llama-3-8B-Instruct

License

Meta Llama 3 Community License. Use of this derivative is bound by the license and acceptable use policy of the original model. Built with Meta Llama 3.