Instructions to use openaccess-ai-collective/mighty-llama-1b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openaccess-ai-collective/mighty-llama-1b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openaccess-ai-collective/mighty-llama-1b", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("openaccess-ai-collective/mighty-llama-1b", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use openaccess-ai-collective/mighty-llama-1b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openaccess-ai-collective/mighty-llama-1b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openaccess-ai-collective/mighty-llama-1b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/openaccess-ai-collective/mighty-llama-1b

SGLang

How to use openaccess-ai-collective/mighty-llama-1b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openaccess-ai-collective/mighty-llama-1b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openaccess-ai-collective/mighty-llama-1b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openaccess-ai-collective/mighty-llama-1b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openaccess-ai-collective/mighty-llama-1b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use openaccess-ai-collective/mighty-llama-1b with Docker Model Runner:
```
docker model run hf.co/openaccess-ai-collective/mighty-llama-1b
```

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Single Pass

hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_accelerate=True,dtype=bfloat16,trust_remote_code=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: 32
|    Task     |Version| Metric |Value |   |Stderr|
|-------------|------:|--------|-----:|---|-----:|
|arc_challenge|      0|acc     |0.2355|_  |0.0124|
|             |       |acc_norm|0.2671|_  |0.0129|
|arc_easy     |      0|acc     |0.4444|_  |0.0102|
|             |       |acc_norm|0.4276|_  |0.0102|
|boolq        |      1|acc     |0.5358|_  |0.0087|
|hellaswag    |      0|acc     |0.3784|_  |0.0048|
|             |       |acc_norm|0.5034|_  |0.0050|
|openbookqa   |      0|acc     |0.1580|_  |0.0163|
|             |       |acc_norm|0.2840|_  |0.0202|
|piqa         |      0|acc     |0.6518|_  |0.0111|
|             |       |acc_norm|0.6464|_  |0.0112|
|winogrande   |      0|acc     |0.5422|_  |0.0140|

16x Passees

hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_accelerate=True,dtype=bfloat16,trust_remote_code=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: 64
|    Task     |Version| Metric |Value |   |Stderr|
|-------------|------:|--------|-----:|---|-----:|
|arc_challenge|      0|acc     |0.2466|_  |0.0126|
|             |       |acc_norm|0.2824|_  |0.0132|
|arc_easy     |      0|acc     |0.3649|_  |0.0099|
|             |       |acc_norm|0.3582|_  |0.0098|
|boolq        |      1|acc     |0.6214|_  |0.0085|
|hellaswag    |      0|acc     |0.3085|_  |0.0046|
|             |       |acc_norm|0.3614|_  |0.0048|
|openbookqa   |      0|acc     |0.1900|_  |0.0176|
|             |       |acc_norm|0.2800|_  |0.0201|
|piqa         |      0|acc     |0.5702|_  |0.0116|
|             |       |acc_norm|0.5729|_  |0.0115|
|winogrande   |      0|acc     |0.5399|_  |0.0140|

Downloads last month: 9