litert-community/codegemma-7b-it-int4-litertlm

CodeGemma 7B IT (INT4 LiteRT LM Model for Android)

This repository contains Google's CodeGemma 7B Instruction-Tuned (google/codegemma-7b-it) quantized to INT4 and exported into the .litertlm model bundle format for on-device deployment on Android using Google's LiteRT runtime (formerly TensorFlow Lite / AI Edge Torch).

---

Model Summary

• Developed by: Google DeepMind / Google AI Edge
• Model Architecture: Gemma Transformer (GemmaForCausalLM)
• Base Checkpoint: google/codegemma-7b-it
• Quantization: INT4 Weight-Only Dynamic Quantization (dynamic_wi4_afp32)
• Export Tool: litert-torch (litert_torch.generative.export_hf)
• Target Platform: Android (ARM64-v8a / Qualcomm Snapdragon / MediaTek / Google Tensor)
• Output File Format: .litertlm (Unified model flatbuffer + tokenizer + prompt templates)

---

Technical Specifications

Parameter	Value
Parameters	7.0 Billion
Hidden Size	3072
Num Layers	28
Num Heads	16
Vocab Size	256,000
Prefill Seq Len	256
Max KV Cache Len	512
Quantization	INT4 (dynamic_wi4_afp32)
File Size	~3.85 GB

---

On-Device Android Integration (LiteRT-LM SDK)

1. Model Deployment

Place the downloaded .litertlm file inside your Android app's assets/ directory or internal storage directory.

2. Dependencies (build.gradle.kts)

dependencies {
    implementation("com.google.ai.edge.litert:litert-lm:1.0.0")
}

3. Execution Code (Kotlin)

import com.google.ai.edge.litert.lm.LiteRtLmEngine
import com.google.ai.edge.litert.lm.ModelOptions
import java.io.File

fun runCodeGemma(context: Context, userPrompt: String): String {
    val modelFile = File(context.filesDir, "codegemma-7b-it-int4.litertlm")
    
    val options = ModelOptions.builder()
        .setModelPath(modelFile.absolutePath)
        .build()

    val engine = LiteRtLmEngine.create(options)
    
    // CodeGemma instruction format is automatically handled by the .litertlm bundle metadata
    val response = engine.generate(userPrompt)
    
    engine.close()
    return response
}

---

Prompt Template Format

The .litertlm bundle automatically packages the official CodeGemma chat template:

<bos><start_of_turn>user
Write a Kotlin function to check prime numbers.<end_of_turn>
<start_of_turn>model

---

Citation & Attribution

If you use this model in your research or applications, please cite the official Google CodeGemma paper and Hugging Face repository:

@article{codegemma2024,
  title={CodeGemma: Open Code Models},
  author={Google CodeGemma Team},
  year={2024}
}

---

Converted using Google LiteRT Torch (litert-torch).