Instructions to use lrds-code/boana-7b-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lrds-code/boana-7b-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lrds-code/boana-7b-instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lrds-code/boana-7b-instruct")
model = AutoModelForCausalLM.from_pretrained("lrds-code/boana-7b-instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use lrds-code/boana-7b-instruct with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lrds-code/boana-7b-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lrds-code/boana-7b-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/boana-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lrds-code/boana-7b-instruct

SGLang

How to use lrds-code/boana-7b-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lrds-code/boana-7b-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/boana-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lrds-code/boana-7b-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/boana-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lrds-code/boana-7b-instruct with Docker Model Runner:
```
docker model run hf.co/lrds-code/boana-7b-instruct
```

lrds-code commited on Jan 23, 2024

Commit

7a45691

verified ·

1 Parent(s): 7550383

Update README.md

Browse files

Files changed (1) hide show

README.md +1 -5

README.md CHANGED Viewed

@@ -30,7 +30,7 @@ tags:
 Boana-7B é um LLM treinado em dados da língua portuguesa. O modelo é baseado no [LLaMA2-7B](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf), uma versão de 7B de parâmetros do LLaMA-2. O projeto Boana tem como objetivo oferecer opções de LLM em língua portuguesa, ao mesmo tempo que disponibiliza um modelo menos complexo para que, dessa forma, usuários com menos poder computacional possam usufruir das LLMs.
-Em apoio aos países de língua portuguesa: Angola, Brasil, Cabo Verde, Guiné Bissau, Guiné Equatorial, Moçambique, Portugal, São Tomé e Príncipe e Timor-Leste.
 <p align="center">
   <img width="250" alt="Countries Logo" src="https://cdn-uploads.huggingface.co/production/uploads/658c21f4c1229bf113295773/d3twZrXng5eDjg_LbH4pF.png">
@@ -121,10 +121,6 @@ In support of portuguese-speaking countries: Angola, Brasil, Cabo Verde, Guinea-
   <img width="250" alt="Countries Logo" src="https://cdn-uploads.huggingface.co/production/uploads/658c21f4c1229bf113295773/d3twZrXng5eDjg_LbH4pF.png">
 </p>
-## Future Updates
-- Add more data from the portuguese language.
 ### Model Description
 - **Developed by:** [Leonardo Souza](https://huggingface.co/lrds-code)

 Boana-7B é um LLM treinado em dados da língua portuguesa. O modelo é baseado no [LLaMA2-7B](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf), uma versão de 7B de parâmetros do LLaMA-2. O projeto Boana tem como objetivo oferecer opções de LLM em língua portuguesa, ao mesmo tempo que disponibiliza um modelo menos complexo para que, dessa forma, usuários com menos poder computacional possam usufruir das LLMs.
+Em apoio aos países de língua portuguesa.
 <p align="center">
   <img width="250" alt="Countries Logo" src="https://cdn-uploads.huggingface.co/production/uploads/658c21f4c1229bf113295773/d3twZrXng5eDjg_LbH4pF.png">
   <img width="250" alt="Countries Logo" src="https://cdn-uploads.huggingface.co/production/uploads/658c21f4c1229bf113295773/d3twZrXng5eDjg_LbH4pF.png">
 </p>
 ### Model Description
 - **Developed by:** [Leonardo Souza](https://huggingface.co/lrds-code)