---

license: apache-2.0

base_model: google/gemma-4-E2B-it

language:

- en

tags:

- aiops

- infrastructure

- kubernetes

- nutanix

- vmware

- active-directory

- pki

- gguf

- mlx

- lora

- gemma4

library_name: gguf

pipeline_tag: text-generation

---

gemma-4-e2b-aiops-gguf

A Q4_K_M GGUF of Gemma 4 E2B, fine-tuned on Apple Silicon (M3 Pro, 18 GB) using MLX LoRA to act as an autonomous AIOps Orchestrator Agent.

The model evaluates multi-domain infrastructure telemetry and outputs strict, execution-ready JSON commands across:

Kubernetes · Nutanix · VMware ESXi · Active Directory · ADFS · PKI · Windows Server

> Cross-platform inference: Trained on macOS (Apple Silicon / Metal), but the GGUF runs on macOS, Linux, and Windows via Ollama or llama.cpp (CUDA, ROCm, Vulkan, or CPU).

---

Model Details

| Property | Value |

|---|---|

| Base model | google/gemma-4-E2B-it |

| Fine-tuning method | LoRA (MLX, 16 layers) |

| Training iterations | 600 |

| Trainable parameters | 6.816M (0.147% of 4.63B) |

| Peak memory (training) | ~11.3 GB on M3 Pro 18 GB |

| Val loss (start → end) | 3.665 → 0.097 |

| Quantization | Q4_K_M (llama.cpp) |

| FP16 size | 9.3 GB |

| Q4_K_M size | 3.2 GB (5.88 BPW) |

| Hardware | Apple M3 Pro, 18 GB Unified Memory |

| Training time | ~5 minutes at ~2.4 it/sec |

---

Quick Start

Ollama (recommended) — macOS · Linux · Windows

# Pull and run directly from this repo
ollama run hf.co/htunn/gemma-4-e2b-aiops-gguf:Q4_K_M

Custom Ollama Agent (with system prompt)

# Modelfile
FROM hf.co/htunn/gemma-4-e2b-aiops-gguf:Q4_K_M

SYSTEM """You are an autonomous AIOps orchestrator agent. Evaluate
telemetry across Kubernetes, Nutanix, VMware, Active Directory, ADFS, and PKI
infrastructure. Output strict, execution-ready JSON commands."""

PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER top_k 40
PARAMETER repeat_penalty 1.1

ollama create aiops-orchestrator -f Modelfile
ollama run aiops-orchestrator

llama.cpp — macOS · Linux · Windows

llama-cli \
  -m gemma4-e2b-aiops-Q4_K_M.gguf \
  --temp 0.3 \
  -p "[AIOps-Agent] Node k8s-worker-03 status is NotReady. AD service account 'svc_k8s_cluster' authentication failed on ADFS."

vLLM (safetensors format)

A HuggingFace safetensors version is available at htunn/gemma-4-e2b-aiops-hf for use with vLLM or the Transformers library:

vllm serve htunn/gemma-4-e2b-aiops-hf --dtype bfloat16

from transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("htunn/gemma-4-e2b-aiops-hf")

---

Example Output

Prompt:

[AIOps-Agent] Node k8s-worker-03 status is NotReady.
Active Directory service account 'svc_k8s_cluster' authentication failed on ADFS.

Response:

{
  "action": "remediate_auth",
  "target_domain": "ADFS",
  "service_account": "svc_k8s_cluster",
  "steps": ["check_pki_cert_validity", "rotate_secret_k8s"],
  "api_call": "POST /api/v1/auth/refresh"
}

---

Prompt:

[AIOps-Agent] PKI certificate for win-srv-2019-app01 expires in 48 hours.
Certificate Authority 'DC-CA-ROOT' is unreachable via LDAP.

Response:

{
  "action": "renew_certificate",
  "target_host": "win-srv-2019-app01",
  "ca_authority": "DC-CA-ROOT",
  "steps": ["verify_ca_health", "request_cert_renewal", "deploy_via_gpo"],
  "priority": "high"
}

---

Prompt:

[AIOps-Agent] Nutanix VM 'win-dc-01' CPU usage exceeds 95%.
VMware host ESXi-02 reports datastore latency.

Response:

{
  "action": "diagnose_hypervisor",
  "target_vm": "win-dc-01",
  "infrastructure": ["nutanix", "vmware"],
  "recommendation": "Live-migrate VM via Prism API to secondary host ESXi-04."
}

---

Training Data Format

Each training example follows the Gemma 4 chat template:

{
  "text": "<bos><start_of_turn>user\n[AIOps-Agent] <telemetry_event><end_of_turn>\n<start_of_turn>model\n{\"action\": \"...\", \"steps\": [...]}<end_of_turn>"
}

Scenarios cover:

• Kubernetes node failures with AD/ADFS auth issues
• Nutanix VM resource exhaustion and live migration
• VMware ESXi host maintenance and cluster recovery
• PKI certificate expiry and CA unreachability
• Windows Server domain controller incidents
• Cross-domain cascading failures

---

Reproduce Fine-Tuning

Source repository: htunn/aiops-gemma4

git clone https://github.com/Htunn/aiops-gemma4.git
cd aiops-gemma4

# Create Python 3.12 venv and install deps
make setup

# Fine-tune (requires HF access to google/gemma-4-E2B-it)
make train

# Fuse + convert to GGUF Q4_K_M
make quantize

# Upload GGUF to HF Hub (Ollama / llama.cpp)
make upload

# Upload safetensors to HF Hub (vLLM / Transformers)
make upload-hf

# Create Ollama model
make ollama

Known mlx-lm Compatibility Notes

• Use --num-layers (not --lora-layers, removed in mlx-lm ≥ 0.28)
• mlx_lm.fuse no longer needs --export-hf (HF format is now default)
• Build llama.cpp with cmake: cmake -B build -DGGML_METAL=ON && cmake --build build --target llama-quantize
• Pin numpy>=2.0, scipy>=1.13, transformers>=5.0 after installing llama.cpp requirements

---

Hardware Requirements

| Task | Memory | Platform |

|---|---|---|

| Fine-tuning (MLX LoRA) | ~11 GB | macOS Apple Silicon |

| Inference Q4_K_M (Ollama / llama.cpp) | ~3.5 GB | macOS · Linux · Windows |

| Inference FP16 (vLLM / Transformers) | ~9.5 GB | Linux (CUDA recommended) |

---

License

Apache 2.0 — see LICENSE.

Base model (google/gemma-4-E2B-it) is subject to the Gemma Terms of Use.