Hugging Face's logo

QuantLLM
/
functiongemma-270m-it-4bit-gguf

GGUF
Transformers
English
quantllm
llama-cpp
quantized
q4_k_m
conversational
Model card Files
xet
Community
functiongemma-270m-it-4bit-gguf
File size: 4,567 Bytes 
aeb30a3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
 
---
license: apache-2.0
base_model: google/functiongemma-270m-it
library_name: gguf
language:
  - en
tags:
  - quantllm
  - gguf
  - llama-cpp
  - quantized
  - transformers
  - q4_k_m
---

<div align="center">

# 🦙 functiongemma-270m-it-4bit-gguf

**google/functiongemma-270m-it** converted to **GGUF** format

[![QuantLLM](https://img.shields.io/badge/🚀_Made_with-QuantLLM-orange?style=for-the-badge)](https://github.com/codewithdark-git/QuantLLM)
[![Format](https://img.shields.io/badge/Format-GGUF-blue?style=for-the-badge)]()
[![Quantization](https://img.shields.io/badge/Quant-Q4_K_M-green?style=for-the-badge)]()

<a href="https://github.com/codewithdark-git/QuantLLM">⭐ Star QuantLLM on GitHub</a>

</div>

---


## 📖 About This Model

This model is **[google/functiongemma-270m-it](https://huggingface.co/google/functiongemma-270m-it)** converted to **GGUF** format for use with llama.cpp, Ollama, LM Studio, and other compatible inference engines.

| Property | Value |
|----------|-------|
| **Base Model** | [google/functiongemma-270m-it](https://huggingface.co/google/functiongemma-270m-it) |
| **Format** | GGUF |
| **Quantization** | Q4_K_M |
| **License** | apache-2.0 |
| **Created With** | [QuantLLM](https://github.com/codewithdark-git/QuantLLM) |


## 🚀 Quick Start

### Option 1: Python (llama-cpp-python)

```python
from llama_cpp import Llama

# Load the model
llm = Llama.from_pretrained(
    repo_id="QuantLLM/functiongemma-270m-it-4bit-gguf",
    filename="functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf",
)

# Generate text
output = llm(
    "Write a short story about a robot learning to paint:",
    max_tokens=256,
    echo=True
)
print(output["choices"][0]["text"])
```

### Option 2: Ollama

```bash
# Download the model
huggingface-cli download QuantLLM/functiongemma-270m-it-4bit-gguf functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf --local-dir .

# Create Modelfile
echo 'FROM ./functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf' > Modelfile

# Import to Ollama
ollama create functiongemma-270m-it-4bit-gguf -f Modelfile

# Chat with the model
ollama run functiongemma-270m-it-4bit-gguf
```

### Option 3: LM Studio

1. Download the `.gguf` file from the **Files** tab above
2. Open **LM Studio** → **My Models** → **Add Model**
3. Select the downloaded file
4. Start chatting!

### Option 4: llama.cpp CLI

```bash
# Download
huggingface-cli download QuantLLM/functiongemma-270m-it-4bit-gguf functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf --local-dir .

# Run inference
./llama-cli -m functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf -p "Hello! " -n 128
```


## 📊 Model Details

| Property | Value |
|----------|-------|
| **Original Model** | [google/functiongemma-270m-it](https://huggingface.co/google/functiongemma-270m-it) |
| **Format** | GGUF |
| **Quantization** | Q4_K_M |
| **License** | `apache-2.0` |
| **Export Date** | 2025-12-21 |
| **Exported By** | [QuantLLM v2.0](https://github.com/codewithdark-git/QuantLLM) |


## 📦 Quantization Details

This model uses **Q4_K_M** quantization:

| Property | Value |
|----------|-------|
| **Type** | Q4_K_M |
| **Bits** | 4-bit |
| **Quality** | 🟢 ⭐ Recommended - Best quality/size balance |

### All Available GGUF Quantizations

| Type | Bits | Quality | Best For |
|------|------|---------|----------|
| Q2_K | 2-bit | 🔴 Lowest | Extreme size constraints |
| Q3_K_M | 3-bit | 🟠 Low | Very limited memory |
| Q4_K_M | 4-bit | 🟢 Good | **Most users** ⭐ |
| Q5_K_M | 5-bit | 🟢 High | Quality-focused |
| Q6_K | 6-bit | 🔵 Very High | Near-original |
| Q8_0 | 8-bit | 🔵 Excellent | Maximum quality |


---

## 🚀 Created with QuantLLM

<div align="center">

[![QuantLLM](https://img.shields.io/badge/🚀_QuantLLM-Ultra--fast_LLM_Quantization-orange?style=for-the-badge)](https://github.com/codewithdark-git/QuantLLM)

**Convert any model to GGUF, ONNX, or MLX in one line!**

```python
from quantllm import turbo

# Load any HuggingFace model
model = turbo("google/functiongemma-270m-it")

# Export to any format
model.export("gguf", quantization="Q4_K_M")

# Push to HuggingFace
model.push("your-repo", format="gguf")
```

<a href="https://github.com/codewithdark-git/QuantLLM">
  <img src="https://img.shields.io/github/stars/codewithdark-git/QuantLLM?style=social" alt="GitHub Stars">
</a>

**[📚 Documentation](https://github.com/codewithdark-git/QuantLLM#readme)** · 
**[🐛 Report Issue](https://github.com/codewithdark-git/QuantLLM/issues)** · 
**[💡 Request Feature](https://github.com/codewithdark-git/QuantLLM/issues)**

</div>