Instructions to use lrds-code/samba-1.1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lrds-code/samba-1.1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lrds-code/samba-1.1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lrds-code/samba-1.1B")
model = AutoModelForCausalLM.from_pretrained("lrds-code/samba-1.1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lrds-code/samba-1.1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lrds-code/samba-1.1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/samba-1.1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lrds-code/samba-1.1B

SGLang

How to use lrds-code/samba-1.1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lrds-code/samba-1.1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/samba-1.1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lrds-code/samba-1.1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/samba-1.1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lrds-code/samba-1.1B with Docker Model Runner:
```
docker model run hf.co/lrds-code/samba-1.1B
```

lrds-code commited on Apr 1, 2024

Commit

4869988

verified ·

1 Parent(s): 6c949e7

Update README.md

Browse files

Files changed (1) hide show

README.md +5 -47

README.md CHANGED Viewed

@@ -17,7 +17,7 @@ widget:
 <hr>
-# PT - README
 <hr>
@@ -104,50 +104,8 @@ alpaca_prompt = f'''Abaixo está uma instrução que descreve uma tarefa, juntam
 {output}'''
 ```
-## Parâmetros importantes
-<hr>
-# EN - README
-<hr>
-# Samba-1.1B
-<p align="center">
-  <img width="250" alt="Samba Logo" src="https://cdn-uploads.huggingface.co/production/uploads/658c21f4c1229bf113295773/MuRvqTWpp-d0NRYQ0yRPL.png">
-</p>
-Samba is a LLM trained on portuguese language data. The model is based on [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0), a 1.1B parameter version of LLaMA-2.
-The LLM Samba project aims to provide more LLM options in Portuguese, while also providing less complex models so that users with less computational power can take advantage of the LLMs.
-<p align="center">
-  <img width="250" alt="Countries Logo" src="https://cdn-uploads.huggingface.co/production/uploads/658c21f4c1229bf113295773/d3twZrXng5eDjg_LbH4pF.png">
-</p>
-### Model Description
-- **Developed by:** [Leonardo Souza](https://huggingface.co/lrds-code)
-- **Model type:** LLaMA-Based
-- **License:** Academic Free License v3.0
-- **Finetuned from model:** [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
-## How to Use
-```python
-import torch
-from transformers import pipeline
-samba = pipeline('text-generation', model='lrds-code/samba-1.1B', torch_dtype=torch.bfloat16, device_map='auto')
-messages = [{"role": "system",
-             "content": ""},
-            {"role": "user",
-             "content": "Quantos planetas existem no sistema solar?"}]
-prompt = samba.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-outputs = samba(prompt, max_new_tokens=256, do_sample=False, temperature=0.1, top_k=50, top_p=0.95)
-print(outputs[0]['generated_text'])
-```

 <hr>
+# README
 <hr>
 {output}'''
 ```
+## Parâmetros Importantes
+- **repetition_penalty:** é utilizado para evitar a repetição de palavras ou frases. Quando esse valor é ajustado para ser maior que 1, o modelo tenta diminuir a probabilidade de gerar palavras que já apareceram anteriormente. Basicamente, quanto maior o valor, mais o modelo tenta evitar repetições.
+- **do_sample:** determina se o modelo deve ou não amostrar aleatoriamente a próxima palavra com base nas probabilidades calculadas. Portanto, **do_sample=True** introduz variação e imprevisibilidade no texto gerado, enquanto que se **do_sample=False** o modelo escolherá sempre a palavra mais provável como próxima palavra, o que pode levar a saídas mais determinísticas e, possivelmente, mais repetitivas.
+- **temperature:** afeta a aleatoriedade na escolha da próxima palavra. Um valor baixo (próximo de 0) faz com que o modelo seja mais "confiante" nas suas escolhas, favorecendo palavras com alta probabilidade e levando a saídas mais previsíveis. Por outro lado, um valor alto aumenta a aleatoriedade, permitindo que o modelo escolha palavras menos prováveis, o que pode tornar o texto gerado mais variado e criativo.