---

license: apache-2.0

base_model: allenai/BAR-2x7B-Tool-Use

tags:

• allenai
• bar
• flex-olmo
• olmo
• moe
• mixture-of-experts
• tool-use
• tool-calling
• gguf
• llama.cpp
• quantized
• q4_k_m
• 12gb

language:

• en

pipeline_tag: text-generation

library_name: gguf

base_model_relation: quantized

---

BAR-2x7B-Tool-Use — GGUF (first-of-its-kind FlexOlmo conversion)

This is the first GGUF conversion of allenai/BAR-2x7B-Tool-Use, one of AllenAI's BAR-family Mixture-of-Experts models released on 2026-04-19 based on the new FlexOlmo architecture.

⚠ Requires a FlexOlmo-enabled llama.cpp

Upstream llama.cpp does not support the FlexOlmo architecture: the support PR

ggml-org/llama.cpp#22540 was closed unmerged on 2026-04-30.

Use hob-llama.cpp, our llama.cpp build that adds FlexOlmo on a current upstream base:

• Prebuilt binaries: release hob-b11434: Linux (CPU x64/arm64, CUDA 12.8, ROCm, Vulkan), Windows (CPU

x64/arm64, CUDA 12.4/13.4, ROCm, Vulkan) and macOS (Intel x64), with SHA256SUMS.txt.

• Or build from source:

git clone https://github.com/Seraphiel102/llama.cpp.git
cd llama.cpp
git checkout hob-b11434
cmake -B build -DGGML_CUDA=OFF        # or -DGGML_CUDA=ON etc. for your GPU
cmake --build build -j --target llama-cli llama-completion llama-server

What FlexOlmo is

Per transformers.models.flex_olmo, FlexOlmoDecoderLayer is Olmo2's hybrid post-norm decoder layer with the dense FFN swapped for OlmoE-style top-k MoE routing. Specifically:

• Attention with q_norm and k_norm (Olmo2-style)
• post_attention_layernorm and post_feedforward_layernorm (post-norm pattern, no input_layernorm)
• Top-k MoE FFN with softmax routing (OlmoE-style)
• No sliding-window attention

Model details

| Field | Value |

|---|---|

| Architecture | FlexOlmoForCausalLM (Olmo2 hybrid + OlmoE MoE) |

| Total parameters | ~11.6 B |

| Active parameters per token | ~11 B (top-2 of 2 experts → effectively dense forward) |

| Layers | 32 |

| Hidden size | 4096 |

| Attention heads | 32 (no GQA; head dim 128) |

| Experts | 2 routed, top-2 selected |

| Vocab | 100,278 |

| Context | 64K (rope_theta 500000) |

Quants

| Quant | Size | Status |

|---|---|---|

| Q4_K_M | ~6.7 GB | ✅ uploaded — recommended for consumer GPUs (12GB+) |

| Q5_K_M | ~8 GB | rolling out (uploaded as ready) |

| Q6_K | ~10 GB | rolling out |

| Q8_0 | ~12 GB | rolling out |

| F16 | ~22 GB | available on request |

Ollama

Ollama's bundled llama.cpp does not support FlexOlmo, so this GGUF does not load in Ollama today. Use hob-llama.cpp (above). The Modelfile in this repo is kept for when Ollama adds support.

Usage — hob-llama.cpp

./build/bin/llama-completion \
    -m BAR-2x7B-Tool-Use.Q4_K_M.gguf \
    -p "Q: A train travels 60 miles in 1.5 hours. What's its average speed in mph?\nA:" \
    -n 60 --temp 0.3

Sample output:

> "To find the average speed, we need to divide the distance by the time. So, the average speed is 60 miles / 1.5 hours = 40 mph."

Validation

Q4_K_M was tested on three prompts during development:

• Factual: "The capital of France is" → "Paris."
• Reasoning: 60mi/1.5h speed problem → "40 mph" with steps
• Code/explanation: fibonacci docstring → coherent explanation of memoization

All clean stops on the model's EOS, no looping or degeneration.

Re-verified 2026-10-06 on hob-llama.cpp b11434: the prompt "The capital of France is" gives byte-identical output and identical top-5 logprobs to the April build (temp 0, 32 tokens).

License

Apache 2.0 (matching the source release at AllenAI). This conversion is a derivative work — same license applies.

Conversion details

• Source: allenai/BAR-2x7B-Tool-Use (downloaded 2026-04-29)
• Tools: our FlexOlmo llama.cpp patch (April 2026); now carried as patch 5 of hob-llama.cpp
• Steps: convert_hf_to_gguf.py → llama-quantize
• Upstream PR ggml-org/llama.cpp#22540 closed unmerged; support lives in hob-llama.cpp

Acknowledgments

• AllenAI for the BAR family release and the FlexOlmo architecture
• llama.cpp maintainers — the existing Olmo2 + OlmoE handlers gave us the right primitives to combine
• This conversion produced for the Zenith swarm project — autonomous engineering collective

Citation

If you use this GGUF, please cite AllenAI's BAR release:

@misc{allenai2026bar,
  title  = {BAR: Beam-and-Adjust-Routing models},
  author = {{Allen Institute for AI}},
  year   = {2026},
  month  = {April},
  url    = {https://huggingface.co/allenai/BAR-2x7B-Tool-Use}
}