---

base_model: OrionLLM/Nebula

library_name: transformers

pipeline_tag: text-generation

license: apache-2.0

language:

• en

tags:

• gguf
• llama-cpp
• text-generation
• reasoning
• nebula
• small-reasoning-model

---

Nebula (320M) GGUF

This repository provides GGUF quantizations for OrionLLM/Nebula.

Nebula is a 320M-parameter generalist Small Reasoning Model trained on 200B+ tokens, designed for edge AI, embedded systems, and on-device deployment. It is optimized for strong memory, general reasoning, arithmetic, and retrieval-friendly tasks while operating with compact compute and memory overhead.

All GGUF files were converted from the original safetensors weights using llama.cpp at native BF16 precision, then quantized into standard k-quant variants.

---

Available Files and Quantizations

| File | Quant Type | Size | Description / Recommendation |

| :--- | :--- | :--- | :--- |

| Nebula-320M-BF16.gguf | BF16 | 614.61 MB | Full precision base conversion. Highest fidelity reference weights. |

| Nebula-320M-Q8_0.gguf | Q8_0 | 327.74 MB | Near-lossless 8-bit quantization. Recommended for best quality. |

| Nebula-320M-Q6_K.gguf | Q6_K | 311.39 MB | High quality retention with minimal degradation. Excellent balance. |

| Nebula-320M-Q5_K_M.gguf | Q5_K_M | 244.94 MB | Solid balance between memory usage and generation accuracy. |

| Nebula-320M-Q4_K_M.gguf | Q4_K_M | 228.60 MB | Fast, lightweight 4-bit quant. Recommended default for mobile and edge setups. |

| Nebula-320M-Q3_K_M.gguf | Q3_K_M | 198.01 MB | Compact footprint when memory headroom is strictly limited. |

| Nebula-320M-Q2_K.gguf | Q2_K | 185.35 MB | Maximum compression. Noticeable quality loss, intended for ultra-constrained environments. |

---

Reasoning Format and Stenographic Tokens

Nebula produces reasoning traces with an intentionally compact, stenographic style designed for efficiency under limited parameter capacity. Traces use structured shorthand notations within <think>...</think> blocks:

Logical Markers

| Token | Meaning | Usage |

| :--- | :--- | :--- |

| → | derivation / implication | Short causal and logical flow |

| ↺ | iterative return / refinement loop | Backtracking, reconsidering priors, RAG re-querying |

| ? | uncertainty / questions to resolve | Appended to short expressions and words |

| !/※ | insight / breakthrough | Marked knowledge discovery |

| ≈ | approximation / estimate | Intermediary hypothesis and preliminary statements |

| ∴ | therefore / final step | Stable conclusion marker |

Uncertainty Markers

| Token | Meaning | Usage |

| :--- | :--- | :--- |

| ● | high confidence | Well-supported empirical or theoretical anchor points |

| ◐ | medium / partial confidence | Incomplete data, plausible but unverified links |

| ○ | low confidence | Speculation, missing context, weak inference chain |

| ⚠ | bias / premise risk | Domain mismatch, cultural assumptions, language-switch artifacts |

| ?maybe? | soft speculation | Tentative branch that may collapse later |

Verification Process

| Token | Meaning | Usage |

| :--- | :--- | :--- |

| ☐ | unverified hypothesis | Raw claim without cross-check |

| ☑ | intermediate verification | Supported by at least one source or argument |

| ✓ | confirmed / validated | Multiple independent supports |

---

Prompt Format

Nebula uses ChatML markup with generation prompts opening into <think> blocks:

<|im_start|>system
You are a helpful reasoning assistant.<|im_end|>
<|im_start|>user
Calculate the square root of 1764 and explain your steps.<|im_end|>
<|im_start|>assistant
<think>

---

How to Use

1. With llama.cpp

Run interactive generation with llama-cli:

llama-cli -m ./Nebula-320M-Q4_K_M.gguf \
    -p "<|im_start|>system\nYou are a helpful reasoning assistant.<|im_end|>\n<|im_start|>user\nWhat is the capital of France and why is it culturally significant?<|im_end|>\n<|im_start|>assistant\n<think>\n" \
    -n 512 \
    -c 4096 \
    --temp 0.6

Launch a local OpenAI-compatible API server:

llama-server -m ./Nebula-320M-Q4_K_M.gguf \
    --host 127.0.0.1 \
    --port 8080 \
    -c 4096

2. With Ollama

Create a file named Modelfile in the same directory:

FROM ./Nebula-320M-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
<think>
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|end_of_text|>"
PARAMETER temperature 0.6

Build and run the model:

ollama create nebula-320m -f Modelfile
ollama run nebula-320m

3. With LM Studio

1. Place the desired .gguf file into your local LM Studio models directory.
2. Select Nebula-320M in the model loader tab.
3. Use the ChatML preset as the prompt template.

---

Model Architecture Details

• Original weights: OrionLLM/Nebula
• Architecture: LlamaForCausalLM (model_type: llama)
• Parameters: ~320M
• Layers: 80
• Hidden size: 576
• Intermediate size: 1536
• Attention heads: 9 query heads, 3 key/value heads (GQA)
• Head dimension: 64
• Context window: 4,096 tokens
• Vocabulary size: 65,536 tokens
• Native precision: bfloat16
• License: Apache 2.0

---

Attribution & Credits

Original model training and architecture by OrionLLM.

GGUF conversions produced by Pink Pixel.

Made with 💖 by Pink Pixel