---

license: apache-2.0

base_model: Qwen/Qwen3.5-2B

tags:

• nl2sh
• natural-language-to-shell
• llama-cpp
• gguf
• lora
• sft

datasets:

• westenfelder/NL2SH-ALFA

language:

• en

pipeline_tag: text-generation

---

nl2sh-qwen3.5-2b

Natural-language → shell command generator, trained as a drop-in alternative to

whatisit-nl2sh's nl2sh-1.5b (Qwen2.5-Coder-1.5B base).

This one uses Qwen3.5-2B (hybrid linear-attention text backbone, vision tower removed for export)

and the same benchmark family for training data.

Ask in plain English:

Q: find files bigger than 100MB in this folder
A: find . -size +100M

Q: list all running docker containers
A: docker ps

Q: watch memory usage update every second
A: watch -n 1 top -b

Files

| file | size | use |

|---|---|---|

| nl2sh-qwen35-q4_k_m.gguf | 1.2G | default pick — smallest practical |

| nl2sh-qwen35-q6_k.gguf | 1.5G | better — statistically indistinguishable from f16 |

| nl2sh-qwen35-q8_0.gguf | 1.9G | near-lossless (+0.07% ppl) |

| nl2sh-qwen35-bf16.gguf | 3.6G | full precision reference |

| nl2sh-qwen35-q4_k_m_imx.gguf | 1.2G | Q4_K_M with imatrix (imatrix.gguf) |

| imatrix.gguf | 1.9M | importance matrix, corpus = NL2SH-ALFA nl/bash pairs |

| lora-adapter/ | 179M | PEFT LoRA adapter used for training |

Recommendation: Q4_K_M if you want the small file (matches what upstream ships at a similar size class);

Q6_K is the sweet spot and costs only +300 MB; Q8_0 if you want effectively zero quantization loss.

Quantization quality (perplexity on held-out NL2SH-ALFA pairs)

| variant | PPL | Δ vs f16 |

|---|---|---|

| f16 | 4.8921 | — |

| bf16 | 4.8919 | −0.00% |

| Q8_0 | 4.8957 | +0.07% |

| Q6_K | 4.9007 | +0.18% |

| Q4_K_M + imatrix | 5.0082 | +2.37% |

| Q4_K_M | 5.0765 | +3.77% |

Greedy-decoding spot checks (16 fixed prompts): Q8_0 matched f16 on 13/16, Q6_K 12/16,

Q4_K_M 9/16 — most differences were equivalent commands rather than errors.

Training setup

Reproducing/forking the upstream recipe, adapted:

| | |

|---|---|

| Base | Qwen3.5-2B (text backbone; vision tower + MTP stripped at export) |

| Data | NL2SH-ALFA train split, 40,639 NL/command pairs (chat format) |

| LoRA rank / alpha / dropout | 32 / 64 / 0.05 |

| Target modules | all linear (incl. GDN linear-attention projections) |

| LR / schedule | 2e-4, cosine, 3% warmup |

| Epochs | 2, packing off |

| Batch | 16 x 2 grad accum (effective 32), seq len 512 |

| Loss | assistant-only (chat-template masked) |

| Precision / seed | bf16, seed 42 |

| Hardware | one RX 9070 XT (gfx1201), ~1h14m |

Final: eval_loss 0.4257, eval token accuracy 87.6%.

Note for RDNA4/ROCm trainers: training required

PYTORCH_HIP_ALLOC_CONF=roundup_power2_divisions:16 and packing disabled to avoid the

gfx1201 hipBLASLt page-fault bug (ROCm#6600).

Usage

llama-server -m nl2sh-qwen35-q4_k_m.gguf --jinja --temp 0

whatisit setup --model ./nl2sh-qwen35-q4_k_m.gguf
whatisit doctor

Generation expects the model's chat template with thinking disabled

(enable_thinking=false → empty <think></think> block, handled automatically by the template).

Greedy decoding (temperature 0), output capped around 64 tokens — it answers with one command.

Benchmark

InterCode-ALFA (300 tasks, temp 0, max_tokens 64) results will be filled in after the harness run.

| model | pass rate |

|---|---|

| GPT-4o (published) | 0.73 |

| nl2sh-3b (upstream) | 0.657 |

| nl2sh-1.5b (upstream, default) | 0.620 |

| this model | pending |

Limitations

• Single-turn, no shell state; English only.
• Output is one command, not a script.
• Trained only on the ALFA split (40k pairs vs upstream's 125k multi-source mix) — broader

data would likely improve coverage on unusual tools.