---

pipeline_tag: text-generation

base_model: beardymcgee/cascamini-llama-350m-edu-adamw-3b-v2

language:

• en

tags:

• gguf
• llama
• causal-lm
• from-scratch
• educational
• research
• cascamini

---

CascaMini-Llama-350M-Edu-AdamW-3B-V2 GGUF

This repository contains the BF16 GGUF release of

beardymcgee/cascamini-llama-350m-edu-adamw-3b-v2.

CascaMini-Llama-350M-Edu-AdamW-3B-V2 is a 340.9M-parameter Llama-style base

language model trained from scratch on approximately 3 billion GPT-2 BPE

tokens presented from a FineWeb-Edu-derived corpus.

It is a research and educational artefact, not an instruction-tuned assistant.

Available file

| File | Format | Notes |

|---|---|---|

| cascamini-llama-350m-edu-adamw-3b-v2-BF16.gguf | BF16 GGUF | Validated release |

Only BF16 is published. A Q8_0 conversion was tested but not released because

its generation behaviour diverged substantially from the validated BF16 model.

Model details

| Field | Value |

|---|---:|

| Parameters | 340,920,320 |

| Architecture | Llama-style dense causal decoder |

| Layers | 24 |

| Hidden size | 1,024 |

| Attention heads | 16 |

| Key/value heads | 4 |

| Context length | 2,048 tokens |

| Tokenizer | GPT-2 BPE, vocabulary size 50,257 |

| Training tokens presented | 3,000,041,472 |

| Fixed validation loss | 2.9834 |

| Fixed validation perplexity | 19.75 |

Run with llama.cpp

llama-cli \
  -m cascamini-llama-350m-edu-adamw-3b-v2-BF16.gguf \
  -ngl 99 \
  -c 2048 \
  -n 160 \
  --temp 0.65 \
  --top-k 40 \
  --top-p 0.90 \
  --min-p 0 \
  --repeat-penalty 1.08 \
  -p "Photosynthesis is the process by which plants"

Recommended sampling settings:

temperature:        0.65
top-k:              40
top-p:              0.90
min-p:              0
repeat penalty:     1.08
new tokens:         100-200

Greedy decoding is not recommended because the model frequently falls into

repetition loops.

Validation

The BF16 GGUF:

• converted successfully from the validated Hugging Face export;
• loaded successfully in llama.cpp;
• preserved the expected tokenizer and model structure;
• generated output with the same general strengths and weaknesses as the

Transformers version.

Limitations

This model:

• is not instruction tuned;
• is not preference or safety tuned;
• frequently produces plausible but incorrect factual statements;
• performs poorly on mathematics and code generation;
• can become repetitive;
• may contradict itself during longer generations;
• should not be used for high-stakes decisions.

The model learned the style and structure of educational text more successfully

than it learned reliable factual knowledge.

Source model

The Transformers/safetensors release, model card, configuration, tokenizer, and

export validation report are available in the source repository:

beardymcgee/cascamini-llama-350m-edu-adamw-3b-v2

Disclaimer

This is an experimental research artefact. Outputs may be incorrect,

misleading, repetitive, or inappropriate.