---

license: apache-2.0

base_model: allenai/BAR-2x7B-Tool-Use

tags:

• allenai
• bar
• flex-olmo
• olmo
• moe
• mixture-of-experts
• tool-use
• tool-calling
• gguf
• llama.cpp
• quantized
• q4_k_m
• 12gb

language:

• en

pipeline_tag: text-generation

library_name: gguf

base_model_relation: quantized

---

BAR-2x7B-Tool-Use — GGUF (first-of-its-kind FlexOlmo conversion)

This is the first GGUF conversion of allenai/BAR-2x7B-Tool-Use, one of AllenAI's BAR-family Mixture-of-Experts models released on 2026-04-19 based on the new FlexOlmo architecture.

⚠ Requires patched llama.cpp

The FlexOlmo architecture is not yet supported in upstream llama.cpp. To run this GGUF you need a build with FlexOlmo support, currently in flight as a PR.

Build from the support branch:

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git checkout feature/flex-olmo-arch  # (or wait for upstream merge)
cmake -B build -DGGML_CUDA=OFF
cmake --build build -j --target llama-cli llama-quantize llama-completion

Once upstream PR lands, any standard llama.cpp / Ollama install will work directly.

What FlexOlmo is

Per transformers.models.flex_olmo, FlexOlmoDecoderLayer is Olmo2's hybrid post-norm decoder layer with the dense FFN swapped for OlmoE-style top-k MoE routing. Specifically:

• Attention with q_norm and k_norm (Olmo2-style)
• post_attention_layernorm and post_feedforward_layernorm (post-norm pattern, no input_layernorm)
• Top-k MoE FFN with softmax routing (OlmoE-style)
• No sliding-window attention

Model details

| Field | Value |

|---|---|

| Architecture | FlexOlmoForCausalLM (Olmo2 hybrid + OlmoE MoE) |

| Total parameters | ~11.6 B |

| Active parameters per token | ~11 B (top-2 of 2 experts → effectively dense forward) |

| Layers | 32 |

| Hidden size | 4096 |

| Attention heads | 32 (no GQA; head dim 128) |

| Experts | 2 routed, top-2 selected |

| Vocab | 100,278 |

| Context | 64K (rope_theta 500000) |

Quants

| Quant | Size | Status |

|---|---|---|

| Q4_K_M | ~6.7 GB | ✅ uploaded — recommended for consumer GPUs (12GB+) |

| Q5_K_M | ~8 GB | rolling out (uploaded as ready) |

| Q6_K | ~10 GB | rolling out |

| Q8_0 | ~12 GB | rolling out |

| F16 | ~22 GB | available on request |

Usage — Ollama

(Once your llama.cpp / Ollama has FlexOlmo support):

hf download Hob-forge/BAR-2x7B-Tool-Use-GGUF \
  BAR-2x7B-Tool-Use.Q4_K_M.gguf Modelfile --local-dir ./bar
cd ./bar
ollama create bar-2x7b-tool:Q4_K_M -f Modelfile
ollama run bar-2x7b-tool:Q4_K_M "Hello"

Usage — llama.cpp (with FlexOlmo support built)

./build/bin/llama-completion \
    -m BAR-2x7B-Tool-Use.Q4_K_M.gguf \
    -p "Q: A train travels 60 miles in 1.5 hours. What's its average speed in mph?\nA:" \
    -n 60 --temp 0.3

Sample output:

> "To find the average speed, we need to divide the distance by the time. So, the average speed is 60 miles / 1.5 hours = 40 mph."

Validation

Q4_K_M was tested on three prompts during development:

• Factual: "The capital of France is" → "Paris."
• Reasoning: 60mi/1.5h speed problem → "40 mph" with steps
• Code/explanation: fibonacci docstring → coherent explanation of memoization

All clean stops on the model's EOS, no looping or degeneration.

License

Apache 2.0 (matching the source release at AllenAI). This conversion is a derivative work — same license applies.

Conversion details

• Source: allenai/BAR-2x7B-Tool-Use (downloaded 2026-04-29)
• Tools: patched llama.cpp on feature/flex-olmo-arch branch
• Steps: convert_hf_to_gguf.py → llama-quantize
• Architecture support added in: PR (pending) <link>

Acknowledgments

• AllenAI for the BAR family release and the FlexOlmo architecture
• llama.cpp maintainers — the existing Olmo2 + OlmoE handlers gave us the right primitives to combine
• This conversion produced for the Zenith swarm project — autonomous engineering collective

Citation

If you use this GGUF, please cite AllenAI's BAR release:

@misc{allenai2026bar,
  title  = {BAR: Beam-and-Adjust-Routing models},
  author = {{Allen Institute for AI}},
  year   = {2026},
  month  = {April},
  url    = {https://huggingface.co/allenai/BAR-2x7B-Tool-Use}
}