This repository contains highly experimental GGUF conversions of [`inclusionAI/Ling-3.0-tiny`](https://huggingface.co/inclusionAI/Ling-3.0-tiny) for an experimental `bailing-hybrid` implementation in llama.cpp.
These files should **not** be treated as stable, production-ready, or numerically certified conversions. The hybrid architecture combines KDA recurrent layers with gated MLA layers and MoE layers, and requires the accompanying experimental llama.cpp source snapshot. A stock llama.cpp checkout may not recognize this architecture.
The quantized files were created directly from the F16 GGUF with llama.cpp's `llama-quantize`. The standard mixed K-quant behavior was used; six small tensors in each quantization required the quantizer's fallback type because their shapes are not compatible with the requested block format.
## Validation performed
- GGUF metadata and tensor descriptors loaded successfully.
- The F16 conversion contains 526 tensors and records the `bailing-hybrid` architecture.
- Q5_K_M and Q4_K_M both loaded successfully in the experimental llama.cpp runtime.
- On an Apple M5, both quantized files offloaded all 25 model layers to `MTL0` and completed a short generation test.
- The build used for validation had `GGML_METAL=ON` with embedded Metal shaders.
Metal support belongs to the llama.cpp runtime rather than the GGUF file itself. Use a build with the experimental architecture changes and Metal enabled if you want GPU offload on Apple hardware.
## Important limitations
This is a **very, very experimental** port. It has not received full long-context, concurrency, state rollback, cross-backend, or exact Hugging Face logits-parity validation. The successful smoke tests are not a guarantee of general correctness or model quality. The F16 file is a reference conversion, not a claim of upstream compatibility.
If stability, reproducibility, or production reliability is important, please use a more established model conversion and a llama.cpp release with official support instead of these files.
The `Ling3.0-tiny-llama.cpp-source-experimental.zip` archive contains the source snapshot used for this conversion, including the local experimental changes. It intentionally excludes `.git`, `.venv`, and generated `build` contents, including `build/bin`.
