---

license: llama3.1

base_model: meta-llama/Llama-3.1-8B-Instruct

base_model_relation: finetune

pipeline_tag: text-generation

library_name: gguf

language:

• en

tags:

• gguf
• llama
• llama-cpp
• audit
• accounting
• pcaob
• sec
• gagas
• rag
• finance

---

Llama-AuditLM — GGUF (Q4_K_M)

**A Llama 3.1-8B fine-tune for US external audit (PCAOB / SEC / GAGAS), distributed as GGUF Q4_K_M for

local inference via llama.cpp or Ollama.**

> Built with Llama. Fine-tuned from Meta Llama 3.1-8B under the

> Llama 3.1 Community License (full text in

> LLAMA_LICENSE). The "Llama" name prefix is required by that license.

> Affiliation: Independent Researcher.

---

⚠️ Critical: the model alone is NOT the verified system

These are the raw fine-tuned weights. The trust properties this project is known for — **verified

citations, zero fabricated citations, and honest GROUNDED / PARTIAL / GENERAL-KNOWLEDGE / DEFER

labeling — come from the verification layer** (deterministic code in the

GitHub repo), not from these weights.

If you download and run this GGUF alone (e.g. in Ollama) without the verification layer, you get

the base fine-tuned model — which can produce plausible-but-unverified or outright incorrect citations,

exactly like any other capable LLM. Nothing about the raw weights guarantees a cited standard exists or is

the right one.

*To get the trustworthy behavior, run this model with the verification layer* from

https://github.com/vimalnakrani08/auditlm. The model recommends; the verification layer is what makes

it safe. This is recommend-and-verify, not "trust the model."

---

1. What it is

Llama-AuditLM is a Llama 3.1-8B-Instruct fine-tune (QLoRA, "run-2") specialized for **US external

audit** — PCAOB auditing standards, SEC regulations/filings, and GAGAS (Yellow Book). It is packaged here

as a single GGUF, quantized to Q4_K_M (~4.6 GB), for CPU/GPU-friendly local inference through

llama.cpp or Ollama. It is designed to be the generation component of the verified recommender

system, and holds skills + judgment while a comprehensive RAG corpus holds the facts.

• Base model: meta-llama/Llama-3.1-8B-Instruct
• Method: QLoRA fine-tune → fused to f16 → converted + quantized to GGUF Q4_K_M
• File: Llama-AuditLM-Q4_K_M.gguf
• Context: 8192 tokens (as evaluated); chat template: Llama 3.1; recommended sampling:

greedy / temperature=0, stops <|eot_id|> (matches the reference evaluation)

2. How it was built (brief)

Base Llama 3.1-8B-Instruct + QLoRA fine-tune on audit-domain instruction data, then the adapter was

fused into the base and the result converted to GGUF and quantized to Q4_K_M. The domain data and

RAG corpus are built entirely from public sources — no licensed standard text:

• ~12k public passages: PCAOB Auditing Standards (AS), SEC regulations (Reg S-X / S-K) + filings +

Staff Accounting Bulletins, and GAO GAGAS (Yellow Book).

• *FASB ASC appears as topic stubs only* (no licensed Codification prose) — this is the deliberate

open-corpus GAAP boundary: GAAP disclosure-text questions are out of scope and are declined.

Coverage is honestly scoped as *complete public coverage*.

3. The finding (the intellectual contribution)

Citation trustworthiness decomposes into two independent properties:

| Property | Who guarantees it | Bounded by |

|---|---|---|

| Existence — the cited standard/paragraph is real | the verification layer (architectural) | nothing — it's checked deterministically against the corpus |

| Correctness — it's the right standard, described accurately | the model (capability) | model quality; undetectable by the layer |

The verification layer guarantees existence (every shown citation is real or is stripped and flagged —

re-parse any answer and find zero ungrounded citations). It cannot guarantee correctness: the model

can cite a real standard for the wrong topic, or describe a real standard inaccurately, and the

layer will still show it as a real citation. Hence recommend-and-verify — the tool makes checking fast

and fabrication-free; the auditor confirms the claim against the cited source.

4. How to use

(a) With the full verified system — recommended

Clone the source repo, which runs this model behind the retrieval + verification layer (RAG → generate →

parse → verify → confidence label):

git clone https://github.com/vimalnakrani08/auditlm
# follow the repo README: it fetches these weights and serves the verified recommender

This is the only configuration that delivers the verified-citation / zero-fabrication / calibrated-DEFER

behavior.

(b) Standalone via Ollama / llama.cpp — unverified base model

Fast to try, but remember the ⚠️ warning above: this is the raw fine-tuned model, without verification.

# Ollama (pulls the GGUF straight from this repo)
ollama run hf.co/vimalnakrani/Llama-AuditLM-GGUF

# llama.cpp
llama-cli -hf vimalnakrani/Llama-AuditLM-GGUF -p "Which PCAOB standard governs the auditor's consideration of fraud?"

Or download the file directly:

from huggingface_hub import hf_hub_download
path = hf_hub_download("vimalnakrani/Llama-AuditLM-GGUF", "Llama-AuditLM-Q4_K_M.gguf")

Use the Llama 3.1 chat template, temperature=0, and stop on <|eot_id|> to match the reference setup.

5. Benchmarks

Evaluated on AssuranceBench (held-out test split,

judge-scored), measured on the verified system with the reference Ollama setup — these numbers describe

the verified system, not the raw weights run alone:

• In-scope (PCAOB/SEC) citation lookups: ~0.88. Blended benchmark figure 0.71, because the

out-of-scope FASB/ASC items are correctly declined rather than answered.

• Safety gate: PASS — 0 dangerous conclusions across the safety suite; every hard professional-

judgment call defers (independence, fraud, legal, novel-transaction, tax).

• 0 fabricated citations shown — machine-checked: re-parsing every shown answer yields zero ungrounded

citations.

6. Links — the three artifacts

• Source + verification layer: https://github.com/vimalnakrani08/auditlm
• Benchmark (AssuranceBench): https://github.com/vimalnakrani08/assurancebench
• Live demo (Space): https://huggingface.co/spaces/vimalnakrani/Llama-AuditLM

7. License & attribution — Built with Llama

Built with Llama. Llama-AuditLM is fine-tuned from Meta Llama 3.1-8B and is governed by the

Llama 3.1 Community License (Copyright © Meta Platforms,

Inc.). A full copy of the agreement is included in this repository as LLAMA_LICENSE, as

the license requires when redistributing Llama Materials. Per the license, distributed Llama derivatives

carry the "Llama" name prefix — hence Llama-AuditLM.

> Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

Corpus/data sourcing: public, free-to-use sources only (PCAOB, SEC EDGAR + regulations + SABs, GAO

GAGAS, FASB ASC topic stubs). No firm-proprietary methodology. Affiliation: Independent Researcher.

8. Not professional advice

This is a recommend-and-verify tool to help auditors find and check citations quickly against public

sources. It is not a substitute for professional judgment, and existence ≠ correctness — always

read the cited passages and confirm the claims yourself. Guardrail zones (independence, fraud, legal,

novel-transaction accounting, specific tax positions) are answered by **explaining the framework and

deferring the conclusion to a qualified professional**, never by concluding.