Instructions to use timinar/baby-llama-58m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use timinar/baby-llama-58m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="timinar/baby-llama-58m", device_map="auto")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("timinar/baby-llama-58m")
model = AutoModelForCausalLM.from_pretrained("timinar/baby-llama-58m", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use timinar/baby-llama-58m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "timinar/baby-llama-58m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "timinar/baby-llama-58m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/timinar/baby-llama-58m

SGLang

How to use timinar/baby-llama-58m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "timinar/baby-llama-58m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "timinar/baby-llama-58m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "timinar/baby-llama-58m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "timinar/baby-llama-58m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use timinar/baby-llama-58m with Docker Model Runner:
```
docker model run hf.co/timinar/baby-llama-58m
```

Baby Llama

Our submission to the strict-small track of the BabyLM challenge.

Baby Llama is a 58M-parameter model, distilled from an ensemble consisting of LLaMA-360M and GPT2-705M, both trained on the babylm_10M dataset.

See the associated paper for a detailed discussion of the training procedure and of the model performance. The training code is available at https://github.com/timinar/BabyLlama.

Hyperparameters for the tasks that require fine-tuning

When evaluating the model on the tasks that require fine-tuning, we noticed that the default hyperparameters suggested by the BabyLM organizers lead to severe overfitting in a number of tasks. To avoid this issue, we have re-tuned those hyperparameters. The sets of hyperparameters selected for each task are listed in the table below.

Task	Maximum learning rate	Batch size	Maximum epochs	Patience	Evaluate every (steps)	Random seed
CoLA	4e-5	64	3	10	20	12
SST-2	5e-5	64	6	10	200	12
MRPC	3e-5	64	3	10	20	12
QQP	4e-5	64	10	10	1000	12
MNLI	5e-5	64	6	10	200	12
MNLI-mm	5e-5	64	6	10	200	12
QNLI	5e-5	64	6	10	200	12
RTE	5e-5	64	6	10	200	12
BoolQ	3e-4	16	10	10	10	12
MultiRC	1e-4	64	7	10	1000	42
WSC	5e-7	1	10	1000	2000	12
CR (Control)	5e-5	64	10	10	100	12
LC (Control)	1e-3	64	1	2	10	12
MV (Control)	5e-5	64	6	10	200	12
RP (Control)	1e-3	64	1	10	10	12
SC (Control)	1e-3	64	2	10	10	12
CR_LC	1e-3	64	2	10	10	12
CR_RTP	5e-5	64	6	10	200	12
MV_LC	5e-5	64	6	10	200	12
MV_RTP	5e-5	64	6	10	200	12
SC_LC	1e-3	64	2	10	10	12
SC_RP	1e-3	64	2	10	10	12

Downloads last month: 4,834

Safetensors

Model size

58.3M params

Tensor type

F32

Space using timinar/baby-llama-58m 1

Paper for timinar/baby-llama-58m

Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty

Paper • 2308.02019 • Published Aug 3, 2023