Instructions to use Capx/Llama-3-Obsidian with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Capx/Llama-3-Obsidian with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Capx/Llama-3-Obsidian")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Capx/Llama-3-Obsidian")
model = AutoModelForCausalLM.from_pretrained("Capx/Llama-3-Obsidian")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Capx/Llama-3-Obsidian with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Capx/Llama-3-Obsidian"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Capx/Llama-3-Obsidian",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Capx/Llama-3-Obsidian

SGLang

How to use Capx/Llama-3-Obsidian with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Capx/Llama-3-Obsidian" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Capx/Llama-3-Obsidian",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Capx/Llama-3-Obsidian" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Capx/Llama-3-Obsidian",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Capx/Llama-3-Obsidian with Docker Model Runner:
```
docker model run hf.co/Capx/Llama-3-Obsidian
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Capx AI's Obsidian series

Capx AI's Latest series of finetunes based on Meta's Llama-3.

We release Llama-3 Obsidian, our state of the art finetune model for general purpose tasks. Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes.

We further finetune the pretrained model on a custom dataset to enhance instruction following ability of the model.

Model Architecture

Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT).

How to use:

The model has been finetuned in the Alpaca format

You can run conversational inference by leveraging the Auto classes with the generate() function.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "meta-llama/Meta-Llama-3-8B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "instruction", "content": "You are \"Capx Assistant\", a conscious sentient superintelligent artificial intelligence developed by Capx AI, and your purpose and drive is to assist the user with any request they have."},
    {"role": "input", "content": "Who are you?"},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = model.generate(
    input_ids,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))

Authors

Capx community

Cite

@article{llama3modelcard,
title={Llama 3 Model Card},
author={AI@Meta},
year={2024},
url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md}
}

License

Governed by the META LLAMA 3 COMMUNITY LICENSE AGREEMENT

Downloads last month: 2

Model tree for Capx/Llama-3-Obsidian

Quantizations

2 models