Add LiquidAgent-1.2B: config, generation config, tokenizer config, and model card

README.md

@@ -0,0 +1,82 @@
+---
+license: other
+library_name: transformers
+pipeline_tag: text-generation
+language:
+- en
+base_model: LiquidAI/LFM2.5-1.2B-Thinking
+tags:
+- lfm2
+- agent
+- fine-tune
+- teichai
+- thinking
+metrics:
+- perplexity
+---
+
+# LiquidAgent-1.2B
+
+A supervised fine-tune of **LiquidAI/LFM2.5-1.2B-Thinking** on OpenAI-style agent
+traces, intended to pick up the agent-trace format and style (thinking + text +
+tool calls). Built for the model request in
+[Compactbot/model-requests#15](https://huggingface.co/spaces/Compactbot/model-requests)
+(requested by @DedeProGames).
+
+> **License note:** This is a derivative of `LiquidAI/LFM2.5-1.2B-Thinking`,
+> which is distributed under the **LFM Open License v1.0** (a custom license;
+> `license: other` on the Hub). That license carries a commercial-use threshold
+> ($10M annual revenue). Read the base model's LICENSE before commercial use.
+
+## What it is
+
+- **Base:** `LiquidAI/LFM2.5-1.2B-Thinking` (LFM2 / `Lfm2ForCausalLM`), bf16.
+- **Params:** 1,170,340,608 (verified by loading the released weights).
+- **Architecture:** 16 blocks — 10 LIV convolutional + 6 full-attention (GQA,
+  32 query / 8 KV heads), `block_dim`/`hidden_size` 2048, `intermediate_size`
+  12288, SwiGLU, RoPE (θ = 1e6), vocab 65536, max position 128000, tied
+  embeddings.
+- **Training:** full fine-tune (not LoRA), bf16, 8-bit AdamW, gradient
+  checkpointing, batch 1, sequence length 32k, cosine LR, 1300 steps.
+- **Data:** 379 agent-trace examples, ~1.34M assistant tokens, packed from
+  `TeichAI/Ox-Alpha-Pi-Traces` + `TeichAI/DeepSeek-v4-Pro-Agent`. Training is
+  masked so the model learns **assistant turns only** (thinking + text + tool
+  calls); user / system / tool-result tokens are masked out.
+
+## Honest quality picture
+
+This is a **light fine-tune**: ~1.34M assistant tokens on top of a 1.2B base.
+It reliably adopts the agent-trace format and thinking style of the source
+traces, but it is not a strong agent and should not be expected to solve
+multi-step agentic tasks reliably.
+
+Verified by loading the released weights standalone and generating:
+- Coherent, grammatical responses with well-formed `[[ctrl:think]]` blocks.
+- Correct simple arithmetic (e.g. 27 × 43 = 1161; 12 + 7 = 19).
+- No degenerate token loops in the sampled prompts.
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Compactbot/liquidagent-1.2b"
+tok = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="cuda")
+
+prompt = "User: What is 27 * 43? Answer briefly."
+ids = tok(prompt, return_tensors="pt").to(model.device)
+out = model.generate(ids, max_new_tokens=256, do_sample=False,
+                     pad_token_id=tok.eos_token_id)
+print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
+```
+
+The chat template is the base model's (with thinking preservation), so
+`apply_chat_template` works as-is.
+
+## Reproducibility
+
+- Base: `LiquidAI/LFM2.5-1.2B-Thinking`
+- Datasets: `TeichAI/Ox-Alpha-Pi-Traces`, `TeichAI/DeepSeek-v4-Pro-Agent`
+- Training: full SFT, bf16, 8-bit AdamW, grad checkpointing, batch 1, seq 32k,
+  cosine LR, 1300 steps, masked to assistant turns.

config.json

@@ -0,0 +1,60 @@
+{
+  "architectures": [
+    "Lfm2ForCausalLM"
+  ],
+  "block_auto_adjust_ff_dim": true,
+  "block_dim": 2048,
+  "block_ffn_dim_multiplier": 1.0,
+  "block_mlp_init_scale": 1.0,
+  "block_multiple_of": 256,
+  "block_norm_eps": 1e-05,
+  "block_out_init_scale": 1.0,
+  "block_use_swiglu": true,
+  "block_use_xavier_init": true,
+  "bos_token_id": 1,
+  "conv_L_cache": 3,
+  "conv_bias": false,
+  "conv_dim": 2048,
+  "conv_use_xavier_init": true,
+  "dtype": "bfloat16",
+  "eos_token_id": 7,
+  "full_attn_idxs": null,
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "layer_types": [
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "full_attention",
+    "conv",
+    "full_attention",
+    "conv",
+    "full_attention",
+    "conv"
+  ],
+  "max_position_embeddings": 128000,
+  "model_type": "lfm2",
+  "norm_eps": 1e-05,
+  "num_attention_heads": 32,
+  "num_heads": 32,
+  "num_hidden_layers": 16,
+  "num_key_value_heads": 8,
+  "pad_token_id": 0,
+  "rope_parameters": {
+    "rope_theta": 1000000.0,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "use_pos_enc": true,
+  "vocab_size": 65536
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 7,
+  "pad_token_id": 0,
+  "transformers_version": "5.5.0"
+}

tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "[[ctrl:startoftext]]",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "[[ctrl:im_end]]",
+  "is_local": true,
+  "legacy": false,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[[ctrl:pad]]",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "use_default_system_prompt": false,
+  "use_fast": true
+}