Instructions to use lrds-code/samba-1.1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lrds-code/samba-1.1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lrds-code/samba-1.1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lrds-code/samba-1.1B")
model = AutoModelForCausalLM.from_pretrained("lrds-code/samba-1.1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lrds-code/samba-1.1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lrds-code/samba-1.1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/samba-1.1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lrds-code/samba-1.1B

SGLang

How to use lrds-code/samba-1.1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lrds-code/samba-1.1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/samba-1.1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lrds-code/samba-1.1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/samba-1.1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lrds-code/samba-1.1B with Docker Model Runner:
```
docker model run hf.co/lrds-code/samba-1.1B
```

lrds-code commited on Jan 13, 2024

Commit

861cf63

verified ·

1 Parent(s): 2faf8c8

Create README.md

Browse files

Files changed (1) hide show

README.md +65 -0

README.md ADDED Viewed

	@@ -0,0 +1,65 @@

+---
+license: afl-3.0
+datasets:
+- ju-resplande/rebel-pt
+- paulofinardi/OIG_small_chip2_portuguese_brasil
+- Guilherme34/Cabrita-lora-ptbr
+- dominguesm/Canarim-Instruct-PTBR-Dataset
+language:
+- en
+- pt
+pipeline_tag: text-generation
+library_name: transformers
+widget:
+  - text: >
+      Pergunta: Quantos planetas existem no sistema solar?
+---
+# Samba-1.1B
+Samba is an LLM trained on portuguese language data. The model is based on [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0), a 1.1B parameter version of LLaMA-2.
+The LLM Samba project aims to provide more LLM options in Portuguese, while also providing less complex models so that users with less computational power can take advantage of the LLMs.
+In support of portuguese-speaking countries. 🇦🇴🇧🇷🇨🇻🇬🇼🇬🇶🇲🇿🇵🇹🇸🇹🇹🇱
+## Model Details
+This model was fine-tuned on two datasets ([rebel-pt](https://huggingface.co/datasets/ju-resplande/rebel-pt) and [OIG_small_chip2_portuguese_brasil](https://huggingface.co/datasets/paulofinardi/OIG_small_chip2_portuguese_brasil) with portuguese data that total approximately 1 million samples.
+## Limitations
+Keep in mind the limitations of this model. It is a model with 1.1B of trained parameters and may present some glitches or hallucinations.
+## Future Updates
+- Add more data from the Portuguese language.
+- Make quantized versions available.
+### Model Description
+- **Developed by:** [Leonardo Souza](https://huggingface.co/lrds-code)
+- **Model type:** LLaMA-Based
+- **License:** Academic Free License v3.0
+- **Finetuned from model:** [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
+## How to Get Started with the Model
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+model = AutoModelForCausalLM.from_pretrained('lrds-code/samba-tiny-llama2-sft')
+tokenizer = AutoTokenizer.from_pretrained('lrds-code/samba-tiny-llama2-sft')
+text = 'Pergunta: Como desenvolver habilidades de programação em python?'
+inputs = tokenizer(text, return_tensors='pt')
+outputs = model.generate(**inputs)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+## Pergunta: Como desenvolver habilidades de programação em python?
+## Resposta: Para desenvolver habilidades de programação em Python, você precisa aprender a ler e escrever código.
+##           Você também precisa entender o que significa cada parte do código e como ela funciona.
+##           Você também precisa entender como usar bibliotecas e frameworks para criar aplicativos.
+##           Além disso, você precisa entender como usar o IDE (Integrated Development Environment) para desenvolver e testar seu código.
+```