Instructions to use Spoon-assassin/functiongemma-270m-it-simple-tool-calling with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Spoon-assassin/functiongemma-270m-it-simple-tool-calling with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Spoon-assassin/functiongemma-270m-it-simple-tool-calling")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Spoon-assassin/functiongemma-270m-it-simple-tool-calling")
model = AutoModelForCausalLM.from_pretrained("Spoon-assassin/functiongemma-270m-it-simple-tool-calling")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Spoon-assassin/functiongemma-270m-it-simple-tool-calling with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Spoon-assassin/functiongemma-270m-it-simple-tool-calling"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Spoon-assassin/functiongemma-270m-it-simple-tool-calling",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Spoon-assassin/functiongemma-270m-it-simple-tool-calling

SGLang

How to use Spoon-assassin/functiongemma-270m-it-simple-tool-calling with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Spoon-assassin/functiongemma-270m-it-simple-tool-calling" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Spoon-assassin/functiongemma-270m-it-simple-tool-calling",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Spoon-assassin/functiongemma-270m-it-simple-tool-calling" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Spoon-assassin/functiongemma-270m-it-simple-tool-calling",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Spoon-assassin/functiongemma-270m-it-simple-tool-calling with Docker Model Runner:
```
docker model run hf.co/Spoon-assassin/functiongemma-270m-it-simple-tool-calling
```

Spoon-assassin commited on Jun 8

Commit

bdf64b0

verified ·

1 Parent(s): cda8bac

Training in progress, step 112

Browse files

Files changed (13) hide show

README.md +5 -5
adapter_config.json +48 -0
adapter_model.safetensors +3 -0
runs/Jun08_12-04-24_45b7d7353d7d/events.out.tfevents.1780920264.45b7d7353d7d.2648.0 +3 -0
runs/Jun08_12-59-06_45b7d7353d7d/events.out.tfevents.1780923546.45b7d7353d7d.16320.0 +3 -0
runs/Jun08_13-06-18_45b7d7353d7d/events.out.tfevents.1780923978.45b7d7353d7d.18354.0 +3 -0
runs/Jun08_13-11-51_45b7d7353d7d/events.out.tfevents.1780924311.45b7d7353d7d.19969.0 +3 -0
runs/Jun08_13-17-24_45b7d7353d7d/events.out.tfevents.1780924644.45b7d7353d7d.21521.0 +3 -0
runs/Jun08_13-38-58_45b7d7353d7d/events.out.tfevents.1780925938.45b7d7353d7d.27315.0 +3 -0
runs/Jun08_13-43-41_45b7d7353d7d/events.out.tfevents.1780926221.45b7d7353d7d.28619.0 +3 -0
tokenizer.json +2 -2
tokenizer_config.json +1 -0
training_args.bin +2 -2

README.md CHANGED Viewed

@@ -4,8 +4,8 @@ library_name: transformers
 model_name: functiongemma-270m-it-simple-tool-calling
 tags:
 - generated_from_trainer
-- trl
 - sft
 licence: license
 ---
@@ -35,10 +35,10 @@ This model was trained with SFT.
 ### Framework versions
-- TRL: 1.3.0
-- Transformers: 5.0.0
-- Pytorch: 2.10.0+cu128
-- Datasets: 4.8.5
 - Tokenizers: 0.22.2
 ## Citations

 model_name: functiongemma-270m-it-simple-tool-calling
 tags:
 - generated_from_trainer
 - sft
+- trl
 licence: license
 ---
 ### Framework versions
+- TRL: 1.5.1
+- Transformers: 5.9.0
+- Pytorch: 2.11.0+cu128
+- Datasets: 5.0.0
 - Tokenizers: 0.22.2
 ## Citations

adapter_config.json ADDED Viewed

	@@ -0,0 +1,48 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/functiongemma-270m-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "down_proj",
+    "gate_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:80d6f4fb461359b2b405f4e8023a15eb2e75de442efb8599870d1d7913e344fc
+size 3829712

runs/Jun08_12-04-24_45b7d7353d7d/events.out.tfevents.1780920264.45b7d7353d7d.2648.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c52157ef2bba0493b49da89bff1f1ab65754b6090f215f3ffd6848fd4b387f7e
+size 4184

runs/Jun08_12-59-06_45b7d7353d7d/events.out.tfevents.1780923546.45b7d7353d7d.16320.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e9f261e1a074ba3b6b5cc33c6833214f9b2267e3dd485aaade4817cfa071654b
+size 6267

runs/Jun08_13-06-18_45b7d7353d7d/events.out.tfevents.1780923978.45b7d7353d7d.18354.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4521c212c82a66b382ebe8fa834adc6b26203c300f9a4796c0045117049c20c3
+size 6266

runs/Jun08_13-11-51_45b7d7353d7d/events.out.tfevents.1780924311.45b7d7353d7d.19969.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a2fb13f3f861c8ab134364f961ac188f8093854ca42ec59f4ace3b6887882dbd
+size 6266

runs/Jun08_13-17-24_45b7d7353d7d/events.out.tfevents.1780924644.45b7d7353d7d.21521.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:55e52141e07e75e05efe5a15b0eaef4cfca63967dd76e81d3ac69d5c80a030ca
+size 6266

runs/Jun08_13-38-58_45b7d7353d7d/events.out.tfevents.1780925938.45b7d7353d7d.27315.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:759507ff42f41d8764a506f8c6738cbab4f24b39a6aadc1485dc03e49d3d8eaf
+size 4184

runs/Jun08_13-43-41_45b7d7353d7d/events.out.tfevents.1780926221.45b7d7353d7d.28619.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e69df67ab57f744ae9df52f3de7b967f3f45ffc652ec09b526ad6c792a47cd3a
+size 51285

tokenizer.json CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e3655797f9d732b7dc08b4225200697af8e37d94b74711d9b1d8166feb953578
-size 33384774

 version https://git-lfs.github.com/spec/v1
+oid sha256:80d7f800b949accd7eb940bac75e642f9468e4df157403032a55bf54ed23b650
+size 33384898

tokenizer_config.json CHANGED Viewed

@@ -7,6 +7,7 @@
   "eos_token": "<eos>",
   "image_token": "<image_soft_token>",
   "is_local": false,
   "mask_token": "<mask>",
   "model_max_length": 1000000000000000019884624838656,
   "model_specific_special_tokens": {

   "eos_token": "<eos>",
   "image_token": "<image_soft_token>",
   "is_local": false,
+  "local_files_only": false,
   "mask_token": "<mask>",
   "model_max_length": 1000000000000000019884624838656,
   "model_specific_special_tokens": {

training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a32f1a3bf5b0e18dded7e4f187016168e810fbbaa1f6dcb737d847c2e9205357
-size 5649

 version https://git-lfs.github.com/spec/v1
+oid sha256:6e782c9b71d826f399de936ae3c3b865bc02c28aee3a10f84b5af247e0f3494d
+size 5777