---

license: other

base_model: Qwen/Qwen3.5-4B

tags:

- gguf

- bf16

- qwen3.5

- merged-lora

- fft-bake

- experimental

language:

- en

- ko

---

LimeCore all-LoRA balanced FFT bake BF16 GGUF

English

This repository contains a BF16 GGUF export of an experimental performance-first dense merge.

The model was built from a Qwen3.5-4B core, multiple specialist LoRA adapters, an all-LoRA balanced merge, and a short full-parameter FFT bake. It is intended as a research artifact for local inference and downstream quantization such as Q4_K_M, Q6_K, or Q8_0.

Current file:

| File | Type | Purpose |

| --- | --- | --- |

| LimeCore-all-4.2B-lora-balanced-fft-bake-BF16.gguf | BF16 GGUF | Source GGUF for local quantization and testing |

Data Summary

No personal identifiers, access tokens, private URLs, or user account details are included in this model card.

The training pipeline used expert-specific cleaned subsets and synthetic expert-control data. The final FFT bake used a smaller distilled text-only mixture from these expert pools.

Text expert preparation:

| Item | Count |

| --- | ---: |

| Source rows scanned | 113,562 |

| Text expert rows selected/written | 57,371 |

| Text selection ratio | 50.52% |

Source categories used during expert preparation:

| Source category | Usage |

| --- | --- |

| General SFT mixture | Repair, Python, math, code, experiment, persona, safety routing |

| Sanitized Lime SFT/alignment data | General repair and assistant behavior cleanup |

| Synthetic reasoning trajectory data | Internal planning, verification, routing, and no-CoT control |

| Synthetic time-series text data | Trend, anomaly, metric-log, and experiment-series reasoning |

| Synthetic dense anchor data | Prompt-conditioned anchor behavior for tags such as [PYTHON_TOOL] and [MATH_REASONING] |

| Vision reasoning data | Used to train the vision adapter; image tensors are not packaged in this text GGUF unless a separate multimodal projection is provided |

Prepared expert training pools:

| Expert pool | Train rows |

| --- | ---: |

| general_repair | 5,478 |

| python_tool | 9,507 |

| math_reasoning | 9,513 |

| code_debug | 9,499 |

| experiment_loop | 9,519 |

| persona_lime | 9,489 |

| safety_control | 1,555 |

| time_series_text_prediction | 5,000 |

| reasoning_trajectory_prediction | 20,000 |

| dense_anchor_alignment | 1,600 |

| vision_reasoning | 9,500 |

Final FFT bake mixture:

| Expert source | Pool rows | Used rows | Pool retention | Bake mix ratio |

| --- | ---: | ---: | ---: | ---: |

| general_repair | 5,478 | 1,000 | 18.25% | 12.05% |

| python_tool | 9,507 | 1,000 | 10.52% | 12.05% |

| math_reasoning | 9,513 | 1,000 | 10.51% | 12.05% |

| code_debug | 9,499 | 1,000 | 10.53% | 12.05% |

| experiment_loop | 9,519 | 800 | 8.40% | 9.64% |

| persona_lime | 9,489 | 400 | 4.22% | 4.82% |

| safety_control | 1,555 | 500 | 32.15% | 6.02% |

| time_series_text_prediction | 5,000 | 800 | 16.00% | 9.64% |

| reasoning_trajectory_prediction | 20,000 | 1,000 | 5.00% | 12.05% |

| dense_anchor_alignment | 1,600 | 800 | 50.00% | 9.64% |

| Total | - | 8,300 | - | 100.00% |

Experimental Techniques

• Sparse expert LoRA training: specialist adapters were trained for Python/tool use, math reasoning, code debugging, experiment loops, safety control, persona behavior, time-series text prediction, reasoning trajectory control, and vision reasoning.
• Performance-first all-LoRA dense merge: all available adapters were merged into a single dense checkpoint with balanced merge scales. This favors broad capability over preserving clean expert separation or a fixed style.
• Qwen3.5-specific anchor alignment: the dense anchor adapter used tags such as [LIME], [PYTHON_TOOL], [MATH_REASONING], [CODE_DEBUG], and [SAFETY]. For Qwen3.5 hybrid attention, the target set included standard attention/MLP projections plus linear-attention projections such as in_proj_qkv, in_proj_z, and out_proj.
• Short full-parameter FFT bake: after merging LoRAs, the dense checkpoint was lightly baked for 100 steps with BF16 weights, low learning rate (5e-6), sequence length 2048, and effective batch size 8. The goal was to stabilize the merged dense model before GGUF export.
• GGUF BF16 export: the baked dense checkpoint was exported to BF16 GGUF as a source artifact for local quantization.

Limitations

• This is an experimental research merge, not a production safety release.
• The model may inherit verbosity, formatting habits, or reasoning-style traces from the expert datasets.
• The BF16 GGUF is intended as a source file for local quantization; Q4/Q6/Q8 behavior should be evaluated separately after quantization.
• Vision adapter training was part of the experiment, but this GGUF file should be treated as a text GGUF unless an accompanying multimodal projection is provided.

한국어

이 저장소는 실험용 성능 우선 dense merge 모델의 BF16 GGUF 내보내기 파일을 담고 있습니다.

이 모델은 Qwen3.5-4B 코어 위에 여러 specialist LoRA를 학습하고, balanced all-LoRA merge를 수행한 뒤, 짧은 full-parameter FFT bake를 거쳐 만든 연구용 산출물입니다. 로컬 추론과 Q4_K_M, Q6_K, Q8_0 같은 후속 양자화를 위한 원본 GGUF로 사용하는 것을 목표로 합니다.

현재 파일:

| 파일 | 형식 | 용도 |

| --- | --- | --- |

| LimeCore-all-4.2B-lora-balanced-fft-bake-BF16.gguf | BF16 GGUF | 로컬 양자화와 테스트를 위한 원본 GGUF |

데이터 요약

이 모델 카드에는 개인정보, 액세스 토큰, 비공개 URL, 사용자 계정 정보가 포함되어 있지 않습니다.

학습 파이프라인은 expert별 정제 데이터와 synthetic expert-control 데이터를 사용했습니다. 최종 FFT bake에는 이 expert pool에서 뽑은 더 작은 text-only distilled mix가 사용되었습니다.

텍스트 expert 정제:

| 항목 | 수량 |

| --- | ---: |

| 스캔한 source row | 113,562 |

| 선택/저장된 text expert row | 57,371 |

| 텍스트 선택 비율 | 50.52% |

사용 데이터 범주:

| 데이터 범주 | 용도 |

| --- | --- |

| General SFT mixture | 일반 수리, Python, 수학, 코드, 실험, persona, safety 라우팅 |

| Sanitized Lime SFT/alignment data | 일반 응답 수리와 assistant behavior 정리 |

| Synthetic reasoning trajectory data | 내부 계획, 검증, 라우팅, no-CoT 제어 |

| Synthetic time-series text data | 추세, 이상치, metric log, 실험 시계열 reasoning |

| Synthetic dense anchor data | [PYTHON_TOOL], [MATH_REASONING] 같은 태그 기반 anchor 동작 |

| Vision reasoning data | vision adapter 학습에 사용; 별도 multimodal projection이 없으면 이 GGUF는 text GGUF로 취급 |

준비된 expert training pool:

| Expert pool | Train rows |

| --- | ---: |

| general_repair | 5,478 |

| python_tool | 9,507 |

| math_reasoning | 9,513 |

| code_debug | 9,499 |

| experiment_loop | 9,519 |

| persona_lime | 9,489 |

| safety_control | 1,555 |

| time_series_text_prediction | 5,000 |

| reasoning_trajectory_prediction | 20,000 |

| dense_anchor_alignment | 1,600 |

| vision_reasoning | 9,500 |

최종 FFT bake mix:

| Expert source | Pool rows | Used rows | Pool retention | Bake mix ratio |

| --- | ---: | ---: | ---: | ---: |

| general_repair | 5,478 | 1,000 | 18.25% | 12.05% |

| python_tool | 9,507 | 1,000 | 10.52% | 12.05% |

| math_reasoning | 9,513 | 1,000 | 10.51% | 12.05% |

| code_debug | 9,499 | 1,000 | 10.53% | 12.05% |

| experiment_loop | 9,519 | 800 | 8.40% | 9.64% |

| persona_lime | 9,489 | 400 | 4.22% | 4.82% |

| safety_control | 1,555 | 500 | 32.15% | 6.02% |

| time_series_text_prediction | 5,000 | 800 | 16.00% | 9.64% |

| reasoning_trajectory_prediction | 20,000 | 1,000 | 5.00% | 12.05% |

| dense_anchor_alignment | 1,600 | 800 | 50.00% | 9.64% |

| Total | - | 8,300 | - | 100.00% |

사용된 실험 기술

• Sparse expert LoRA 학습: Python/tool, math reasoning, code debugging, experiment loop, safety control, persona behavior, time-series text prediction, reasoning trajectory control, vision reasoning용 specialist adapter를 학습했습니다.
• 성능 우선 all-LoRA dense merge: 모든 사용 가능한 adapter를 balanced scale로 하나의 dense checkpoint에 병합했습니다. 이는 expert 분리나 고정된 말투 보존보다 전체 성능을 우선하는 실험입니다.
• Qwen3.5 전용 anchor alignment: dense anchor adapter는 [LIME], [PYTHON_TOOL], [MATH_REASONING], [CODE_DEBUG], [SAFETY] 같은 태그를 사용했습니다. Qwen3.5의 hybrid attention 구조를 고려해 일반 attention/MLP projection 외에도 in_proj_qkv, in_proj_z, out_proj 같은 linear-attention projection을 target에 포함했습니다.
• 짧은 full-parameter FFT bake: LoRA 병합 후 BF16 weight, 낮은 learning rate(5e-6), sequence length 2048, effective batch size 8 조건으로 100 step bake를 수행했습니다. 목적은 GGUF export 전에 merge된 dense 모델을 안정화하는 것입니다.
• GGUF BF16 export: bake된 dense checkpoint를 로컬 양자화용 source artifact로 BF16 GGUF 변환했습니다.

한계

• 이 모델은 production safety release가 아니라 실험용 research merge입니다.
• expert 데이터의 영향으로 장황함, 특정 포맷 습관, reasoning-style 흔적이 나타날 수 있습니다.
• BF16 GGUF는 로컬 양자화를 위한 원본 파일입니다. Q4/Q6/Q8 결과는 양자화 후 별도로 평가해야 합니다.
• vision adapter 학습은 실험에 포함되었지만, 별도 multimodal projection이 제공되지 않는 한 이 GGUF는 text GGUF로 취급해야 합니다.