Instructions to use RaoFoundation/774M-03_09_2024 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RaoFoundation/774M-03_09_2024 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RaoFoundation/774M-03_09_2024")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RaoFoundation/774M-03_09_2024")
model = AutoModelForCausalLM.from_pretrained("RaoFoundation/774M-03_09_2024")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RaoFoundation/774M-03_09_2024 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RaoFoundation/774M-03_09_2024"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RaoFoundation/774M-03_09_2024",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/RaoFoundation/774M-03_09_2024

SGLang

How to use RaoFoundation/774M-03_09_2024 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RaoFoundation/774M-03_09_2024" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RaoFoundation/774M-03_09_2024",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RaoFoundation/774M-03_09_2024" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RaoFoundation/774M-03_09_2024",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use RaoFoundation/774M-03_09_2024 with Docker Model Runner:
```
docker model run hf.co/RaoFoundation/774M-03_09_2024
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

This model was trained for Bittensor's pretraining subnet (https://github.com/RaoFoundation/pretraining) and, as of March 9th 2024, is the top model.

Bittensor subnet 9 rewards miners for producing pretrained Foundation-Models on the Falcon Refined Web dataset. It acts like a continuous benchmark whereby miners are rewarded for attaining the best losses on randomly sampled pages of Falcon given a consistent model architecture. The reward mechanism works as follows:

Miners train and periodically publish models to hugging face and commit the metadata for that model to the Bittensor chain.
Validators download the models from hugging face for each miner based on the Bittensor chain metadata and continuously evaluate them, setting weights based on the performance of each model against the Falcon dataset. They also log results to wandb.
The Bittensor chain aggregates weights from all active validators using Yuma Consensus to determine the proportion of TAO emission rewarded to miners and validators.

Bittensor hosts multiple incentive mechanism through which miners are evaluated by validators for performing actions well. Validators perform the process of evaluation and 'set weights', which are transactions into Bittensor's blockchain. Each incentive mechanism in Bittensor is called a 'subnet' and has an identifier (This particular mechanism has subnet uid 9). Weights and the amount of TAO held by the validators become inputs to Bittensor's consensus mechanism called Yuma Consensus. YC drives validators towards a consensus, agreement about the value of the work done by miners. The miners with the highest agreed upon scores are minted TAO, the network digital currency.

Miners within this subnet are evaluated based on the number of times the model they have hosted has a lower loss than another model on the network when randomly sampling from the near infinite Falcon Refined Web pretraining dataset. To perform well, miners must attain the lowest loss on the largest number of random batches. Finding the best model and delta at the earliest block ensures the most incentive.

Downloads last month: 86

Safetensors

Model size

0.8B params

Tensor type

F32

Model tree for RaoFoundation/774M-03_09_2024

Quantizations

2 models

RaoFoundation
/

774M-03_09_2024

Model tree for RaoFoundation/774M-03_09_2024

Dataset used to train RaoFoundation/774M-03_09_2024