Instructions to use Konstantinos/el_llama_smol with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Konstantinos/el_llama_smol with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Konstantinos/el_llama_smol")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Konstantinos/el_llama_smol")
model = AutoModelForCausalLM.from_pretrained("Konstantinos/el_llama_smol")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Konstantinos/el_llama_smol with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Konstantinos/el_llama_smol"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Konstantinos/el_llama_smol",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Konstantinos/el_llama_smol

SGLang

How to use Konstantinos/el_llama_smol with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Konstantinos/el_llama_smol" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Konstantinos/el_llama_smol",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Konstantinos/el_llama_smol" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Konstantinos/el_llama_smol",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Konstantinos/el_llama_smol with Docker Model Runner:
```
docker model run hf.co/Konstantinos/el_llama_smol
```

language: el

el-llama-smol

Model:

el-llama-smol aims to be the first in a series of LLMs trained mostly in Greek corpora. The model is a small (1bn parameters) version of LLama, with the following configuration.

{
  "architectures": ["LLaMAForCausalLM"],
  "bos_token_id": 0,
  "eos_token_id": 1,
  "hidden_act": "silu",
  "hidden_size": 2048,
  "intermediate_size": 5461,
  "initializer_range": 0.02,
  "max_sequence_length": 1024,
  "model_type": "llama",
  "num_attention_heads": 32,
  "num_hidden_layers": 24,
  "pad_token_id": -1,
  "rms_norm_eps": 1e-06,
  "transformers_version": "4.28.1",
  "use_cache": true,
  "vocab_size": 22000
}

Training details:

The current snapshot has been trained for 40hrs with an RTX A6000 GPU (48G), using the galore_adamw8bit_per_layer optimizer by Zhao et. al [1] and a context size of 1024 tokens.

Dataset:

The model is trained on the Greek subset of the allenai/c4 dataset. Text tokenization is performed with a (heavily unoptimized) tokenizer with vocab size of 22000 tokens, trained with SentencePiece

Examples

Use a 🤗 pipeline


from transformers import pipeline
pipe = pipeline("text-generation", model="Konstantinos/el_llama_smol")

set_seed(1)
prompt = """Η Ιαπωνία έχει μια ιστορία που ξεκινά πριν από χιλιάδες χρόνια. 
Οι επιστήμονες πιστεύουν πως οι Ιάπωνες ως ενιαίο σύνολο προέρχονται από πολλές ομάδες,
οι οποίες μετανάστευσαν στα νησιά από άλλα σημεία της Ασίας, στα οποία περιλαμβάνονται """

ret = pipe(prompt, do_sample=True, top_k=20, temperature=0.85,  max_new_tokens=110)

Load model directly


from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Konstantinos/el_llama_smol")
model = AutoModelForCausalLM.from_pretrained("Konstantinos/el_llama_smol")

References

[1] Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, & Yuandong Tian. (2024). GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection.

Citation

TBD

license: odc-by

Downloads last month: 6