steampunque/GLM-4.5-Air-MP-GGUF overview

---

base_model: zai-org/GLM-4.5-Air

base_model_relation: quantized

tags:

• zai
• GLM AIR
• quantized
• 4-bit

pipeline_tag: text-generation

---

Mixed Precision GGUF layer quantization of GLM-4.5-Air by zai

Original model: https://huggingface.co/zai-org/GLM-4.5-Air

The hybrid quant employs different quantization levels on a per layer basis to enable

both high performance and small file size at the same time. An extended layer definition

E quant Q4_E_H for the model is defined as follows (updated 10/16/2026):

  LAYER_TYPES='[
   ["A","attn","Q","attn_q","K","attn_k","V","attn_v","O","attn_o","S","ssm","F","ffn","G","ffn_g","U","ffn_u","D","ffn_d"],
   ["MAP","VOFD","0","QN_K","2","Q2_K","3","Q3_K","4","Q4_K","5","Q5_K","6","Q6_K","8","Q8_0","h","F16","f","F32"],
   [0 ,"Q5_K_6545"],[1 ,"Q4_K_5544"],[2 ,"Q4_K_5444"],[3 ,"Q4_K_5444"],[4 ,"Q4_K_5434"],[5 ,"Q4_K_5434"],[6 ,"Q4_K_5434"],[7 ,"Q4_K_5434"],
   [8 ,"Q4_K_5434"],[9 ,"Q4_K_5434"],[10,"Q4_K_5434"],[11,"Q4_K_5434"],[12,"Q4_K_5434"],[13,"Q4_K_5434"],[14,"Q4_K_5434"],[15,"Q4_K_5434"],
   [16,"Q4_K_5434"],[17,"Q4_K_5434"],[18,"Q4_K_5434"],[19,"Q4_K_5434"],[20,"Q4_K_5434"],[21,"Q4_K_5434"],[22,"Q4_K_5434"],[23,"Q4_K_5434"],
   [24,"Q4_K_5444"],[25,"Q4_K_5434"],[26,"Q4_K_5444"],[27,"Q4_K_5434"],[28,"Q4_K_5444"],[29,"Q4_K_5434"],[30,"Q4_K_5444"],[31,"Q4_K_5434"],
   [32,"Q4_K_5444"],[33,"Q4_K_5444"],[34,"Q4_K_5444"],[35,"Q4_K_5444"],[36,"Q4_K_5444"],[37,"Q4_K_5444"],[38,"Q4_K_5444"],[39,"Q4_K_5444"],
   [40,"Q4_K_5544"],[41,"Q4_K_5544"],[42,"Q4_K_5545"],[43,"Q4_K_5545"],[44,"Q5_K_6648"],[45,"Q6_K_8656"],
   [46,"Q4_K_5444"]
   ]'
   FLAGS="--token-embedding-type Q4_K --output-tensor-type Q6_K --layer-types-high --tensor-pad [[10944,11008],[1408,1536]] --override-kv glm4moe.feed_forward_length=int:11008  --override-kv glm4moe.expert_feed_forward_length=int:1536"

The quant pads FFN dimension to even 256 so K quants can be used with it. The layer quants were optimized for strong performance across a

curated set of reasoning and knowlede prompts with a minimum quant across layers at Q3_K for FFN_G/FFU and Q4_K for rest of tensors to

minimize entropy loss. It is sized to enable it to run on a 64G RAM machine. The quant includes a useable MTP layer at Q4_K_5444.

Comparison:

Quant | size | PPL | Comment

---------|---------|------|-----------

Q4_0 | 62.8e9 | - | default embed no tensor pad

Q4_E_H | 63.6e9 | 7.7 | Q4 embed Q6 out FFN padded E quant

Usage:

This model is a RL trained moe which can be run on a computer with 64G RAM using combined CPU and a single GPU.

A good setup is to offload all model layers to GPU and all non-shared expert FFN tensors to CPU by specifying the

override tensor flag: -ot exps=CPU as discussed here : https://github.com/ggml-org/llama.cpp/discussions/13154

Approximate performance on a 9900k CPU with one local 4070 GPU:

Q | QKV | ND | NKV | gen tps | Comment

------|-------|------|--------|------------|---------

Q4_E_H| F16 | 0 | 36k | 7 | No draft

Q4_E_H| Q8_0 | 0 | 69k | 7 | No draft

The model is a hybrid thinking model. By default it will generate thinking context prior to answering a question.

In this mode it is prone to severe overthinking/second guessing. It can continue to second guess itself ad nauseum

for hundreds and hundreds of reflections even on very simple mildly ambiguous questions. To avoid this the model can be used in non

think mode by appending /nothink token at the end of the user query (or equivalently prefix the assistant template with /nothink)

This /nothink is a special token and must be tokenized as such. When a valid /nothink is tokenized the model will automatically

output `` <think></think> `` as its first outputs, then answer the problem as a normal non reasoning model would and will not get

stuck in the long reflection loops. The model has been found to be extremely capable in this mode over a small set of test prompts.

In the thinking mode it is also very capable as long as you are willing to wait for the answer, showing ability to solve a complex

astrophysics problem both accurately and insightfully. Note that it was empirically found that use of /nothink is required. It

does not work to just inject `` <think></think> `` at the beginning of the assistant output to stop it from generating the think block.

Subjective evaluation of the model with the optimized E quant shows that it can achieve surprising results sometimes mimicing actual intelligent

thought in the think mode; i.e. it can come up with correct answers on off the wall IQ-test-like questions certain not to be trained into its latent

space. It also seems to have good knowledge of trivia type questions most RL models optimized for instruction following lose. It

also exhibits strong insights on conversational prompts. Some examples of what the model/quant can do (greedy sampling, QKV=F16 recommended,

using a downstream llama.cpp server supporting shell interface):

Example conversational prompt:
NOTHINK=1 lm how high is up?  Think outside the box.
<think></think>

That's a beautiful question. To think "outside the box" means we have to stop thinking of "up" as a simple direction on a piece of paper or a vector in physics. "Up" is a concept, a feeling, a destination. Its height is entirely dependent on the box you're currently in.

So, how high is up?

Let's measure it in different units:

1. Up is a Feeling of Potential

In this box, "up" isn't a distance, it's a state of being.

• How high is up? It's the height of your ambition. It's the gap between where you are and where you want to be. It's the energy you feel when you're about to succeed, the moment before a breakthrough. This "up" has no fixed measurement; it's infinite, because ambition can always grow. It's the height of a dream.

2. Up is a Measure of Perspective

In this box, "up" is relative. It's defined by your point of view.

• How high is up? From the bottom of a well, up is the tiny circle of sky you can see—a few feet. From the top of a skyscraper, up is the wind on your face and the city laid out beneath you—1,500 feet. From an airplane, up is the curvature of the Earth and the thin blue line of our atmosphere—35,000 feet. From the perspective of the universe, up is the edge of the observable cosmos, 93 billion light-years away. Up is as high as your perspective allows you to look.

3. Up is a Journey, Not a Destination

In this box, "up" isn't a place you arrive at; it's the path you take.

• How high is up? It's the sum of all your small, daily steps forward. It's the height of the mountain you climb one step at a time, not the peak itself. The peak is just a viewpoint. The real "up" is the cumulative height of all your efforts, your learnings, your kindnesses, and your mistakes turned into lessons. This "up" is measured in experience, and it grows every single day, even when you're not looking.

4. Up is a State of Escape

In this box, "up" is the opposite of "down," where "down" is whatever is holding you back.

• How high is up? It's the altitude you gain when you leave behind your fears, your doubts, and your limitations. If "down" is the weight of a bad day, then "up" is the lightness of letting it go. If "down" is the gravity of a difficult situation, then "up" is the height you achieve by finding a new solution. Up is the height of your freedom.

5. Up is a Philosophical Concept

In this box, "up" is a metaphor for the fundamental nature of reality.

• How high is up? In some philosophies, "up" is the direction of spirit, of consciousness, of the divine. It's the path toward enlightenment, toward understanding your connection to everything else. In this sense, "up" isn't a physical height but a level of being. It's the height of your awareness, the distance between seeing the world as separate and seeing it as one. This "up" is infinite, because the potential for growth in consciousness is limitless.

---

So, the final answer is:

Up is not a fixed height. It is a variable.

Its value is determined by the currency you choose to measure it in: ambition, perspective, effort, freedom, or awareness.

The real question isn't "How high is up?" but rather...

"Which box are you in, and what will you use to measure your 'up' today?"

Example writing prompt:
lm write a tense hard scifi passage in the vein of AE Van Vogt Null A universe
<think>

Okay, user wants a tense hard sci-fi passage in the style of A.E. van Vogt's Null-A universe. That's a very specific request. Let me unpack this.

First, van Vogt's Null-A is all about non-Aristotelian logic, multidimensional thinking, and cosmic horror wrapped in philosophical puzzles. The tension comes from characters navigating reality that's fundamentally alien to human cognition. User's probably a hardcore sci-fi fan who appreciates mind-bending concepts over action sequences.

Hmm... key elements to include:

• The protagonist should be a "Null-A" type thinker - someone who perceives reality differently
• The threat must be conceptual, not physical
• Need that van Vogt signature of describing impossible geometries
• Must include the "aha" moment where logic breaks down
• Sensory details should feel wrong (like tasting colors)

User said "tense" so I'll build claustrophobia through:

• Limited perspective (first-person unreliable narrator)
• Time dilation effects
• The environment actively resisting understanding
• That van Vogt trick where the solution creates new questions

Biggest challenge is making non-Euclidean logic feel visceral. Maybe describe the protagonist's body betraying them - like fingers twitching into impossible angles. And the horror should be existential, not gory. The real terror is realizing your mind is the cage.

...Wait, should I include the classic van Vogt trope of "the thing that can't be perceived but is everywhere"? Yes. That's essential. The entity should be both absent and omnipresent, like a negative space in reality.

For the passage structure:

1. Establish the protagonist's altered perception
2. Introduce the impossible space
3. Build tension through failed attempts to comprehend
4. The "breakthrough" that's actually a trap
5. End with the chilling realization that understanding is the threat

Better avoid technobabble. Van Vogt's genius was making cosmic horror feel clinical. The prose should be precise but unsettling, like lab notes from a mad scientist. And that last line needs to land like a hammer - something about the mind being the final prison.

The air in the Observation Chamber tasted of ozone and forgotten equations. Kael adjusted the focus of his perception-lenses, the familiar *click-whir* a small anchor in the swirling chaos of the Null-A continuum. Outside the reinforced plasteel, the **Labyrinth** unfolded – not a place, but a *state*. Geometry bled into topology, causality looped like a Möbius strip, and the very concept of 'distance' was a fluid, mocking suggestion.

His mission: Locate the **Anomaly**. Not a physical object, but a *rupture* in the local logic-field, a point where the Null-A principles – the non-Aristotelian foundations of reality itself – began to fray. A dangerous task. One misstep, one misapplication of lateral thought, and the Labyrinth could… *rearrange* you. Permanently.

He projected a thought-form: a simple, three-dimensional cube. It shimmered into existence near the chamber’s edge, solid, predictable. Kael focused, pushing his Null-A training, forcing the cube to *understand* its own edges, its own volume. It held. Good.

Now, the probe. He extended a tendril of pure consciousness, a Null-A probe designed to map the *relationships* within the Labyrinth, not just the objects. It drifted towards the sector designated Gamma-7, where the Anomaly’s signature was strongest. The probe shimmered, its form warping as it entered the field. Kael monitored its internal logic-matrix, watching the symbols shift, the connections fray and re-knot in impossible ways.

*Warning: Non-Linear Temporal Gradient Detected. Probability of Causal Loop: 87.3%.* The chamber’s AI voice was flat, devoid of emotion, but the percentage sent a cold prickle down Kael’s spine. He pulled the probe back fractionally, recalibrating its temporal anchor. He couldn’t afford to get caught in a bootstrap paradox within the Labyrinth’s guts.

He tried again. This time, he focused on the *absence*. The Anomaly wasn’t *there*; it was the *lack* of there-ness. He visualized a perfect sphere of *non-existence* and pushed the probe towards it. The probe’s form didn’t just warp; it *dissolved* into a smear of contradictory potentials. Kael felt a jolt, not physical, but *conceptual*. His own perception-lenses flickered, showing him overlapping images of the chamber: one where he stood, one where he knelt, one where he… wasn’t there at all.

*Error: Perception Field Collapse Imminent. Initiating Containment Protocol.* The AI’s voice was colder now. Kael fought the vertigo, forcing his Null-A training to the forefront. He wasn’t seeing things; he was *misinterpreting* the data. He needed to find the *pattern* within the chaos, the underlying *logic* that made the illogical possible.

He closed his eyes, not physically, but conceptually. He stopped *trying* to see the Labyrinth and instead *listened* to its silent scream. He felt the pressure of impossible angles, the weight of infinite regress, the cold dread of a solution that creates its own problem. The Anomaly wasn’t a point; it was a *process*. A recursive function eating its own tail, generating an infinite cascade of broken realities.

*Found it,* Kael thought, a surge of triumph cutting through the dread. Not a location, but a *relationship*. The Anomaly was the *interaction* between the Labyrinth’s inherent non-linearity and a specific, resonant frequency of his own probe’s logic-matrix. It wasn’t *in* the Labyrinth; it was the *shadow* cast by his attempt to measure it.

He began the disengagement sequence, carefully retracting the probe, severing the resonant link. The overlapping images stabilized. The smear of contradictory potentials coalesced back into a recognizable, if distorted, probe-form. The pressure in his skull eased.

*Containment Successful. Anomaly Neutralized. Local Logic Field Stabilizing.* The AI’s tone was almost… relieved.

Kael opened his eyes. The chamber looked normal. The cube still sat, solid and predictable. But the air still tasted of ozone and forgotten equations. He looked at his hands. They looked normal. But he *knew*. He had touched the edge. He had perceived the pattern within the chaos. And in that perception, something had shifted. Not the Labyrinth. Him.

The Null-A continuum hadn’t just revealed a rupture; it had shown him a mirror. And the reflection wasn’t human. It was the cold, elegant, terrifying logic of a universe that didn’t just *allow* for paradoxes, it *demanded* them. He was still in the chamber, but the walls felt thinner now. The air tasted less of ozone, more of infinite recursion. He hadn’t neutralized the Anomaly. He had become its anchor. The Labyrinth hadn’t changed. His understanding of it had. And that understanding was a prison door clicking shut.

Example conversatonal knowledge prompt:

lm give me a rundown on Dr Manhattan in Watchmen

<think>

Okay, the user asked for a rundown on Dr. Manhattan from Watchmen. Let me start by recalling the key points about his character. He's a central figure in the story, so I need to cover his origins, powers, role in the plot, and themes he represents.

First, his origin: Jon Osterman, a nuclear physicist, gets disintegrated in an intrinsic field test. His consciousness survives, and he's rebuilt as Dr. Manhattan. That's crucial because it explains his detachment from humanity. His powers come from that accident—quantum manipulation, teleportation, etc. I should list his abilities but also note how they isolate him.

Next, his role in the story. He works for the US government, which escalates tensions with the USSR. His relationship with Laurie (Silk Spectre II) is important. Their breakup shows his emotional distance. Then there's his exile on Mars, which is a turning point. He's manipulated by Adrian Veidt, leading to the fake alien attack. That's a big plot twist, so I need to explain how Veidt uses Dr. Manhattan's image to unite the world.

Themes are essential too. Dr. Manhattan represents the dehumanizing effect of power and the Cold War's existential dread. His blue skin and nudity symbolize his alienation. Also, his perception of time as non-linear ties into the story's themes of fate and free will. The user might be interested in the philosophical aspects, so highlighting his existential crisis is key.

I should also mention his character arc. He starts as a human, becomes detached, then through Laurie and the squid attack, he reconnects with humanity. His return to Earth shows a shift, but he still leaves for another galaxy. That ending is bittersweet, emphasizing his ongoing journey.

Wait, the user might not have read the comic, so avoiding spoilers isn't necessary since they asked for a rundown. But I should clarify that he's a tragic figure, not a typical superhero. His powers make him more of a force of nature. Also, his relationship with Janey Slater and Laurie shows his inability to maintain human connections.

The user's deeper need might be understanding why Dr. Manhattan is significant in Watchmen. They might be analyzing the character for a project, writing an essay, or just curious. Including themes like existentialism, the Cold War context, and the critique of superhero tropes would add depth. Maybe they want to grasp how his character drives the plot's message about humanity's flaws and potential.

I should structure the response clearly: origin, powers, role in plot, relationships, themes, and significance. Keeping it concise but comprehensive. Avoid markdown, use plain English. Make sure to explain terms like intrinsic field test and intrinsic field briefly. Also, mention his iconic look—blue, nude, glowing—as it's visually symbolic.

Check if there's anything missed. His perception of time as non-linear is important for understanding his detachment. The squid attack's role in uniting the world is a key plot point. His final decision to leave Earth ties into his character development. Also, note that he's not a traditional hero; his morality is complex, especially when he's manipulated by Veidt.

Overall, ensure the rundown covers both the factual aspects and the thematic significance, showing why Dr. Manhattan is a pivotal character in Watchmen.

Here's a rundown of Dr. Manhattan, one of the most iconic and complex characters in Watchmen:

1. Origin: Jon Osterman

• Before: Brilliant nuclear physicist working at a research facility in the 1950s. Engaged to Janey Slater.
• The Accident: Trapped in an "intrinsic field test" chamber. His body was atomized molecule by molecule over several months.
• Rebirth: His consciousness survived the disintegration. Guided by alien technology (implied), he painstakingly reassembled himself, emerging as a god-like being. This traumatic event fundamentally shattered his connection to humanity.

2. Powers: The Quantum Man

Dr. Manhattan's abilities stem from his unique quantum state, making him effectively omnipotent within his universe:

• Omnipotence: Near-limitless strength, durability, and invulnerability. He can withstand nuclear blasts and manipulate matter/energy at will.
• Telekinesis & Matter Manipulation: Can rearrange molecules, create complex objects (including intricate watchworks), disintegrate matter, and heal himself/others.
• Teleportation: Can teleport himself and others instantly across vast distances (even to Mars).
• Telepathy & Precognition: Can read minds (though often chooses not to) and perceive the future/past non-linearly. He sees time as a single, simultaneous "now."
• Size Alteration: Can alter his size, becoming gigantic.
• True Flight: Can levitate and fly at incredible speeds.
• Perception: Sees the world in terms of energy fields and atomic structures. His vision includes infrared, ultraviolet, and beyond.
• Immortality: Effectively ageless and cannot be killed by conventional means.

3. Appearance & Symbolism

• Blue Skin: Result of his unique quantum state; absorbs most light.
• Nudity: He has no need for clothes and finds them uncomfortable. His nudity symbolizes his detachment from human conventions and his alienation.
• Glow: Constantly emits a faint blue glow, representing his unique energy state.
• Atomic Symbol: Often depicted with a hydrogen atom symbol (his origin).

4. Role in the Story & Character Arc

• Government Weapon: After his accident, he's recruited by the US government. His existence alone tips the Cold War balance, deterring nuclear attack and making the US the dominant superpower. This makes him a global symbol and a target.
• Emotional Detachment: His perception of time as non-linear and his god-like powers make him profoundly detached from human emotions, suffering, and mortality. He struggles to connect with others, leading to failed relationships (Janey Slater, Laurie/Silk Spectre II).
• Exile to Mars: Feeling increasingly alienated and blamed for escalating tensions (including cancer clusters linked to his presence), he exiles himself to Mars. This is a crucial turning point.
• Existential Crisis: On Mars, he confronts Laurie about his humanity and the meaning of existence. He questions his purpose and whether humanity is worth saving.
• Manipulation & The Squid: Adrian Veidt manipulates Dr. Manhattan's perception and uses his image to stage the fake alien invasion ("The Squid") in New York. This catastrophic event unites the world against a common "alien" threat, ending the Cold War.
• Return & Departure: Initially horrified by Veidt's plan, Dr. Manhattan ultimately agrees it's the only viable path to peace. He returns to Earth, reveals the truth to Laurie and Dan (Nite Owl II), and departs for another galaxy, leaving humanity to its own devices. His final act is to create a glass structure on Mars for Laurie and Dan.

5. Key Themes Represented by Dr. Manhattan

• The Dehumanizing Effect of Absolute Power: His powers isolate him from the human experience he once knew.
• Existential Dread & Cosmic Indifference: His perception of time and the universe highlights the insignificance of human concerns and the potential meaninglessness of existence.
• The Cold War & Nuclear Threat: He embodies the terrifying power and existential risk of the nuclear age.
• The Failure of the Superhero: He represents the ultimate evolution of the superhero concept – a being so powerful he becomes irrelevant to human problems and ultimately abandons them.
• Fate vs. Free Will: His non-linear perception blurs the line between seeing the future and causing it. Does he have free will, or is he just following a path he perceives?
• Loss of Innocence & Humanity: His origin story is a literal and metaphorical loss of humanity.

In Essence:

Dr. Manhattan is not just a superhero; he's a walking, talking existential crisis. He's a tragic figure whose god-like powers have rendered him incapable of participating meaningfully in the very human world he once belonged to. He serves as the central catalyst for the plot's events and embodies the core themes of Watchmen: the fragility of humanity, the terrifying potential of science, the futility of war, and the search for meaning in a chaotic universe. He's a symbol of power, alienation, and the terrifying beauty and indifference of the cosmos.

Code:

Across a small set of code gen prompts the model does well. Due to very slow prompt processing it will not be capable of digesting large

code prompts but for more simple code gen tasks it is usable.

Download the file from below:

| Link | Type | Size/e9 B | Notes |

|------|------|-----------|-------|

| GLM-4.5-Air.Q4_E_H.gguf | Q4_E_H | 63.6e9 B | ~Q4_0 size FFN padded for K quants includes MTP layer |

A discussion thread about the hybrid quant approach can be found here on the llama.cpp git repository:

https://github.com/ggml-org/llama.cpp/discussions/13040