Text Generation
Transformers
Safetensors
English
keylm75m
keylm
small-language-model
instruct
gqa
rope
swiglu
qk-norm
custom_code
conversational
Instructions to use Eclipse-Senpai/KeyLM-75M-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Eclipse-Senpai/KeyLM-75M-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Eclipse-Senpai/KeyLM-75M-Instruct", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Eclipse-Senpai/KeyLM-75M-Instruct", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use Eclipse-Senpai/KeyLM-75M-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Eclipse-Senpai/KeyLM-75M-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Eclipse-Senpai/KeyLM-75M-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Eclipse-Senpai/KeyLM-75M-Instruct
- SGLang
How to use Eclipse-Senpai/KeyLM-75M-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Eclipse-Senpai/KeyLM-75M-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Eclipse-Senpai/KeyLM-75M-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Eclipse-Senpai/KeyLM-75M-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Eclipse-Senpai/KeyLM-75M-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Eclipse-Senpai/KeyLM-75M-Instruct with Docker Model Runner:
docker model run hf.co/Eclipse-Senpai/KeyLM-75M-Instruct
Update README.md
Browse files
README.md
CHANGED
|
@@ -29,7 +29,7 @@ datasets:
|
|
| 29 |
|
| 30 |
# KeyLM-75M-Instruct
|
| 31 |
|
| 32 |
-
KeyLM-75M-Instruct is a 75M parameter instruction-tuned language model trained from scratch on approximately 18 billion tokens. That training budget is a small fraction of what comparable small models use (SmolLM-135M was trained on
|
| 33 |
|
| 34 |
## Table of Contents
|
| 35 |
|
|
@@ -60,8 +60,6 @@ GGUF builds for `llama.cpp`, LM Studio, and Ollama are available at [KeyLM-75M-I
|
|
| 60 |
|
| 61 |
## How to Use
|
| 62 |
|
| 63 |
-
KeyLM ships its own modeling code, so load it with `trust_remote_code=True`. It requires `transformers>=4.51`.
|
| 64 |
-
|
| 65 |
```python
|
| 66 |
import torch
|
| 67 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
@@ -83,8 +81,6 @@ outputs = model.generate(
|
|
| 83 |
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
|
| 84 |
```
|
| 85 |
|
| 86 |
-
The model uses a plain `User:` / `Assistant:` chat format, applied automatically by `apply_chat_template`. Assistant turns end with `</s>`.
|
| 87 |
-
|
| 88 |
## Evaluation
|
| 89 |
|
| 90 |
### Instruction following (IFEval)
|
|
|
|
| 29 |
|
| 30 |
# KeyLM-75M-Instruct
|
| 31 |
|
| 32 |
+
KeyLM-75M-Instruct is a 75M parameter instruction-tuned language model trained from scratch on approximately 18 billion tokens. That training budget is a small fraction of what comparable small models use (SmolLM-135M was trained on 600B tokens and SmolLM2-135M on 2T). Despite this, it is competitive on instruction following, outperforming SmolLM-135M-Instruct on IFEval while using about half the parameters and a fraction of the data.
|
| 33 |
|
| 34 |
## Table of Contents
|
| 35 |
|
|
|
|
| 60 |
|
| 61 |
## How to Use
|
| 62 |
|
|
|
|
|
|
|
| 63 |
```python
|
| 64 |
import torch
|
| 65 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 81 |
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
|
| 82 |
```
|
| 83 |
|
|
|
|
|
|
|
| 84 |
## Evaluation
|
| 85 |
|
| 86 |
### Instruction following (IFEval)
|