jeffasante commited on
Commit
ab5ebab
·
verified ·
1 Parent(s): d443958

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  gemma-4-2p3b-it/gemma-4-2p3b-it-litert.cellm filter=lfs diff=lfs merge=lfs -text
 
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  gemma-4-2p3b-it/gemma-4-2p3b-it-litert.cellm filter=lfs diff=lfs merge=lfs -text
37
+ gemma-4-2p3b-it-litert/gemma-4-2p3b-it-litert.cellm filter=lfs diff=lfs merge=lfs -text
gemma-4-2p3b-it-litert/README.md ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: gemma
3
+ base_model: google/gemma-4-2p3b-it
4
+ library_name: cellm
5
+ tags:
6
+ - mobile
7
+ - rust
8
+ - memory-efficient
9
+ - quantized
10
+ - gemma
11
+ - gemma-4
12
+ - litert
13
+ datasets: []
14
+ metrics:
15
+ - latency
16
+ - throughput
17
+ pipeline_tag: text-generation
18
+ ---
19
+
20
+ # Gemma-4 2.3B IT for Cellm
21
+
22
+ This repository contains the **Gemma-4 2.3B Instruct** model optimized for the [cellm](https://github.com/jeffasante/cellm) memory-efficient serving engine.
23
+
24
+ ## Model Details
25
+ - **Architecture**: Gemma-4
26
+ - **Parameters**: 2.3B
27
+ - **Format**: `.cellm` (Native cellm distribution format)
28
+ - **Size**: ~2.4 GB
29
+
30
+ ## Usage
31
+
32
+ ### Prerequisites
33
+ You need the [cellm](https://github.com/jeffasante/cellm) engine built from source.
34
+
35
+ ### Run Inference
36
+ ```bash
37
+ ./target/release/infer \
38
+ --model gemma-4-2p3b-it-litert.cellm \
39
+ --tokenizer tokenizer.json \
40
+ --prompt "Write one short sentence about Rust programming." \
41
+ --chat \
42
+ --gen 24 \
43
+ --backend metal
44
+ ```
45
+
46
+ ## License
47
+ This model is subject to the **Gemma Terms of Use**. By downloading or using these weights, you agree to the terms listed at [ai.google.dev/gemma/terms](https://ai.google.dev/gemma/terms).
gemma-4-2p3b-it-litert/gemma-4-2p3b-it-litert.cellm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3bec9b01e9d1c25678f9691b8759bf56f67716039f576b5b86a9fe50563eba34
3
+ size 2538864640