Replace broken int4 bundle with a working int8 build; rewrite card with measured numbers

Thanks for adding a CodeGemma bundle to the community catalog — I ran it as part of a pass running every catalog bundle locally on a Mac.

The published int4 file generates repetitive word-salad on both backends ("sophistic sophistic ..." on CPU, ';" ;"' on GPU; litert-lm 0.15.0 and 0.16.0, file sha256-verified against the Hub). I tried to rebuild int4 from google/codegemma-7b-it and both attempts (channelwise and block-32, two toolchain versions) collapse the same way — this checkpoint doesn't survive data-free int4 PTQ. An int8 export does generate correctly, so this PR replaces the broken file with that build and rewrites the card to match what's inside: measured M4 Max numbers, the desktop-class size stated plainly, and a note on why int4 isn't offered. Generation was verified on CPU and GPU before benchmarking.

If you'd rather keep this repo int4-only or handle it differently, happy to adjust — the working file is the main thing.

README.md

@@ -1,111 +1,64 @@
 ---
 license: gemma
 base_model: google/codegemma-7b-it
-library_name: litert
+base_model_relation: quantized
+library_name: litert-lm
 tags:
 - litert
+- litert-lm
 - litertlm
 - codegemma
-- int4
-- android
 - on-device
 pipeline_tag: text-generation
 ---
 
-# CodeGemma 7B IT (INT4 LiteRT LM Model for Android)
+# CodeGemma 7B IT (LiteRT-LM)
 
-This repository contains Google's **CodeGemma 7B Instruction-Tuned (`google/codegemma-7b-it`)** quantized to **INT4** and exported into the **`.litertlm`** model bundle format for on-device deployment on Android using Google's **LiteRT** runtime (formerly TensorFlow Lite / AI Edge Torch).
+[google/codegemma-7b-it](https://huggingface.co/google/codegemma-7b-it) in the
+LiteRT-LM `.litertlm` format for local inference with
+[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM).
 
----
-
-## Model Summary
-
-- **Developed by**: Google DeepMind / Google AI Edge
-- **Model Architecture**: Gemma Transformer (`GemmaForCausalLM`)
-- **Base Checkpoint**: [`google/codegemma-7b-it`](https://huggingface.co/google/codegemma-7b-it)
-- **Quantization**: INT4 Weight-Only Dynamic Quantization (`dynamic_wi4_afp32`)
-- **Export Tool**: `litert-torch` (`litert_torch.generative.export_hf`)
-- **Target Platform**: Android (ARM64-v8a / Qualcomm Snapdragon / MediaTek / Google Tensor)
-- **Output File Format**: `.litertlm` (Unified model flatbuffer + tokenizer + prompt templates)
-
----
+| File | Quantization | Size | Notes |
+|---|---|---:|---|
+| `codegemma-7b-it-int8.litertlm` | dynamic int8 (weights int8, activations fp32) | 8.6 GB | KV cache 4096 tokens |
 
-## Technical Specifications
+This is a desktop-class file: at 8.6 GB it needs a machine with ample RAM
+(tested on Apple silicon); it does not fit 8 GB-RAM phones.
 
-| Parameter | Value |
-|---|---|
-| **Parameters** | 7.0 Billion |
-| **Hidden Size** | 3072 |
-| **Num Layers** | 28 |
-| **Num Heads** | 16 |
-| **Vocab Size** | 256,000 |
-| **Prefill Seq Len** | 256 |
-| **Max KV Cache Len** | 512 |
-| **Quantization** | INT4 (`dynamic_wi4_afp32`) |
-| **File Size** | ~3.85 GB |
+**Why int8 and not int4:** codegemma-7b-it collapses under data-free int4
+post-training quantization — int4 builds (channelwise and block-32, two
+different toolchain versions) all produce repetitive word-salad on every
+backend. The int8 build generates correctly. If you need this model smaller,
+it will take a calibration-based method, not a recipe change.
 
----
-
-## On-Device Android Integration (LiteRT-LM SDK)
-
-### 1. Model Deployment
-Place the downloaded `.litertlm` file inside your Android app's `assets/` directory or internal storage directory.
+## Usage
 
-### 2. Dependencies (`build.gradle.kts`)
-```kotlin
-dependencies {
-    implementation("com.google.ai.edge.litert:litert-lm:1.0.0")
-}
+```bash
+pip install litert-lm
+litert-lm run codegemma-7b-it-int8.litertlm \
+  --prompt "Write a Python function that returns the square of a number."
 ```
 
-### 3. Execution Code (Kotlin)
-```kotlin
-import com.google.ai.edge.litert.lm.LiteRtLmEngine
-import com.google.ai.edge.litert.lm.ModelOptions
-import java.io.File
+The bundle embeds CodeGemma's chat template (`<start_of_turn>user … <end_of_turn>`),
+so plain prompts work; no manual formatting needed.
 
-fun runCodeGemma(context: Context, userPrompt: String): String {
-    val modelFile = File(context.filesDir, "codegemma-7b-it-int4.litertlm")
-    
-    val options = ModelOptions.builder()
-        .setModelPath(modelFile.absolutePath)
-        .build()
+## Performance (Apple M4 Max, measured)
 
-    val engine = LiteRtLmEngine.create(options)
-    
-    // CodeGemma instruction format is automatically handled by the .litertlm bundle metadata
-    val response = engine.generate(userPrompt)
-    
-    engine.close()
-    return response
-}
-```
+Measured with the LiteRT-LM CLI: `litert-lm benchmark -p 256 -d 256 --runs 3
+--cache no` (litert-lm 0.15.0) on an idle Apple M4 Max (macOS); 256 prefill /
+256 decode tokens, 3 iterations averaged by the tool.
 
----
+| Backend | Prefill (tokens/s) | Decode (tokens/s) | Time-to-first-token (s) |
+|---|---|---|---|
+| CPU | 239.1 | 9.1 | 1.18 |
+| GPU | 455.7 | 29.7 | 0.60 |
 
-## Prompt Template Format
+Generation was verified on both backends before quoting numbers (a coding
+prompt returns a correct, well-formed function).
 
-The `.litertlm` bundle automatically packages the official CodeGemma chat template:
+## Conversion
 
-```text
-<bos><start_of_turn>user
-Write a Kotlin function to check prime numbers.<end_of_turn>
-<start_of_turn>model
-```
-
----
-
-## Citation & Attribution
-
-If you use this model in your research or applications, please cite the official Google CodeGemma paper and Hugging Face repository:
-
-```bibtex
-@article{codegemma2024,
-  title={CodeGemma: Open Code Models},
-  author={Google CodeGemma Team},
-  year={2024}
-}
-```
-
----
-*Converted using Google LiteRT Torch (`litert-torch`).*
+Exported with [litert-torch](https://github.com/google-ai-edge/litert-torch)
+0.9.3 (`python -m litert_torch.generative.export_hf`, `--cache_length 4096`;
+weights come out as dynamic int8 with fp32 activations) from the original
+safetensors checkpoint (sha256-verified download).

codegemma-7b-it-int4-litertlm.litertlm → codegemma-7b-it-int8.litertlm

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:895b4ef975512d67d3a9160305f1e9d7525bf87bbfa01ec33225325b35983f5b
-size 4724446544
+oid sha256:d17210866e31d9444179ed13de8c4f2a49655c62c4aaa36e8f2dba672f0b969c
+size 8596756992