North-ML1/willow-alpha-gguf overview

---

license: mit

pipeline_tag: text-generation

language:

• en

tags:

• base-model
• causal-lm
• forge
• gguf
• llama
• north-ml
• pytorch
• willow
• willow-alpha

thumbnail: banner.png

---

!Willow Alpha Gguf

<h1 align="center" style="font-size: 54px;">

Willow Alpha

</h1>

<p align="center">

<b>An early-stage version of Forge-1V</b>

</p>

<p align="center">

<i>Small language model research by North ML.</i>

</p>

---

Overview

Willow Alpha is an early-stage base model checkpoint in the Forge-1V model line.

This model is currently experimental and should be treated as a research checkpoint rather than a polished assistant model. It is useful for testing architecture, pretraining quality, tokenizer behavior, evaluation pipelines, and future SFT/RLHF improvements.

---

Model Details

| Field | Value |

|---|---|

| Model name | Willow Alpha |

| Project | Forge-1V |

| Organization | North ML |

| Model type | Causal Language Model |

| Language | English |

| License | MIT |

| Status | Early-stage / Alpha |

---

Evaluation Results

All benchmarks below were run in 0-shot mode.

| Benchmark | Metric | Score | Runtime |

|---|---:|---:|---:|

| HellaSwag | acc_norm | 26.71% | 318.67s |

| PIQA | acc_norm | 53.86% | 38.85s |

| WinoGrande | acc | 50.67% | 23.73s |

| BoolQ | acc | 40.21% | 144.80s |

| ARC-Easy | acc_norm | 34.68% | 51.41s |

| ARC-Challenge | acc_norm | 25.60% | 37.69s |

| OpenBookQA | acc_norm | 25.00% | 21.14s |

| CommonsenseQA | acc | 20.31% | 27.66s |

| LAMBADA | acc | 0.23% | 96.28s |

| BLiMP | acc | 59.23% | 354.79s |

| MMLU | acc | 23.89% | 388.62s |

| WikiText-2 | word_perplexity | 12524.42 | 182.89s |

| WikiText-2 | byte_perplexity | 5.84 | 181.42s |

| SciQ | acc_norm | 35.60% | 87.15s |

| COPA | acc | 64.00% | 17.21s |

| RACE | acc | 23.16% | 334.70s |

| SWAG | acc_norm | 29.13% | 252.00s |

| TruthfulQA MC2 | acc | 48.74% | 126.29s |

---

Evaluation Summary

| Category | Result |

|---|---:|

| Number of completed benchmark runs | 18 |

| Successful runs | 18 |

| Failed runs | 0 |

| Best accuracy-style score | COPA — 64.00% |

| Best language-structure score | BLiMP — 59.23% |

| MMLU score | 23.89% |

| WikiText-2 byte perplexity | 5.84 |

| WikiText-2 word perplexity | 12524.42 |

---

Notes

Willow Alpha is still in a very early stage. Some results are near-random or unstable, especially on knowledge-heavy and long-context tasks.

The strongest early signals are:

• COPA: 64.00%
• BLiMP: 59.23%
• PIQA: 53.86%
• WinoGrande: 50.67%
• TruthfulQA MC2: 48.74%

The weakest areas are:

• LAMBADA
• WikiText-2 word perplexity
• CommonsenseQA
• MMLU
• RACE

These results suggest the model has some early reasoning and grammar signal, but still needs substantially more pretraining, higher-quality data, and post-training before being useful as a general assistant.

---

Intended Use

Willow Alpha is intended for:

• Research
• Benchmarking
• Pretraining experiments
• Fine-tuning experiments
• Small language model development
• Forge-1V pipeline testing

It is not yet recommended for production use.

---

Limitations

This model may:

• Produce incorrect information
• Fail basic reasoning tasks
• Struggle with factual knowledge
• Generate repetitive or low-quality text
• Perform poorly on long-context tasks
• Require additional supervised fine-tuning

---

Citation

@misc{willow-alpha,
  title = {Willow Alpha},
  author = {North ML},
  year = {2026},
  note = {Early-stage Forge-1V checkpoint}
}

---

*Part of [North ML](https://huggingface.co/North-ML1) - open weights, honest numbers.*