Upload README.md with huggingface_hub

README.md

@@ -1,22 +1,40 @@
----
-base_model: unsloth/Llama-3.2-1B-Instruct
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- latent-reasoning
-- colar
-- research
----
-# colar-coding-l1b
-
-single-track CoLaR, coding 域(符号执行/coding_mix), warm-start 自 colar-gsm, compress=5
-
-- **Base model:** `unsloth/Llama-3.2-1B-Instruct`
-- **Files:** `colar_coding.ckpt`, `hparams.yaml`
-
-## Loading (PyTorch-Lightning checkpoint — NOT AutoModel-loadable)
-Weights live under the top-level key `['state_dict']` and only fit the custom CoLaR scaffold (base LLM + `[PAD]` resize + r128 q/v LoRA + a `LatentPolicy` MLP), loaded `strict=False`. Load the base separately and splice this `state_dict` in. Runtime env:
-```
-COLAR_BASE=<base>  COLAR_CKPT=colar-gsm/colar_best.ckpt  COLAR_EMB_STD=0.018  COLAR_COMPRESS=5  COLAR_MAXLAT=64  TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1
-```
-`TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1` is required for these older Lightning ckpts.
+---
+base_model: unsloth/Llama-3.2-1B-Instruct
+language:
+- en
+license: llama3.2
+library_name: colar
+pipeline_tag: text-generation
+tags:
+- latent-reasoning
+- colar
+- reasoning
+- research
+---
+# CoLaR — Coding (symbolic-execution reasoning), Llama-3.2-1B
+
+A **CoLaR** (Compressed Latent Reasoning) checkpoint for **Coding (symbolic-execution style reasoning)**, fine-tuned from `unsloth/Llama-3.2-1B-Instruct`. The model reasons in *compressed continuous latent embeddings* rather than explicit chain-of-thought tokens.
+
+## Model
+- **Base model:** `unsloth/Llama-3.2-1B-Instruct`
+- **Framework:** CoLaR — *Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains* (arXiv:2505.16552).
+- **Domain:** Coding (symbolic-execution style reasoning)
+- **Warm-start:** colar-gsm (GSM8K CoLaR base)
+- **Files:** `colar_coding.ckpt`, `hparams.yaml`
+
+## Training procedure
+Trained with the CoLaR supervised fine-tuning (SFT) recipe: the frozen base LLM is adapted with **q/v LoRA (rank 128, alpha 32)** plus a trainable **Latent Head** (3-layer MLP) and an **embedding-compression** module. The objective is next-token **cross-entropy** on the answer plus an **`embed_modeling_loss` (MSE)** that reconstructs the compressed reasoning-step embeddings (each latent token summarizes about `compression_factor` chain-of-thought tokens). Reinforcement learning (GRPO) is **disabled** — this is an SFT-only checkpoint.
+- **This checkpoint:** CoLaR SFT, compression factor 5, warm-started from colar-gsm; SFT only (RL off).
+
+## Datasets
+- Coding reasoning mix (symbolic-execution style)
+
+## How to load
+This is a **PyTorch-Lightning checkpoint** (weights under the top-level key `state_dict`) that fits the CoLaR scaffold — it is **not** directly `AutoModel`-loadable. Load the base model, splice this `state_dict` in with `strict=False`, and use the CoLaR runtime settings:
+```
+COLAR_EMB_STD=0.018   COLAR_COMPRESS=<compression_factor>   sep_token=###
+TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1
+```
+See the CoLaR repository (github.com/xiaomi-research/colar) for the exact loader. The shared GSM8K warm-start ancestor is the official `AlbertTan/CoLaR` release.
+
+*Research artifact for latent-reasoning study (small 1–1.5B model).*