Instructions to use RaghuCourage9605/Anubis-559M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RaghuCourage9605/Anubis-559M with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="RaghuCourage9605/Anubis-559M", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("RaghuCourage9605/Anubis-559M", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use RaghuCourage9605/Anubis-559M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "RaghuCourage9605/Anubis-559M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RaghuCourage9605/Anubis-559M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/RaghuCourage9605/Anubis-559M
- SGLang
How to use RaghuCourage9605/Anubis-559M with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "RaghuCourage9605/Anubis-559M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RaghuCourage9605/Anubis-559M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "RaghuCourage9605/Anubis-559M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RaghuCourage9605/Anubis-559M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use RaghuCourage9605/Anubis-559M with Docker Model Runner:
docker model run hf.co/RaghuCourage9605/Anubis-559M
Model Card for Anubis-559M
Model Description
Anubis-559M is a causal language model designed for creative text generation. It is a fine-tuned version of a larger base model, adapted to excel at generating natural-sounding dialogue and engaging prose. The model is particularly well-suited for role-playing scenarios and creative writing, aiming to produce text that feels human-written.
Intended Uses & Potential Limitations
Intended Use
This model is intended for use in a variety of creative text generation applications, including:
- Role-playing and Storytelling: Generating dialogue and narrative for interactive fiction and games.
- Creative Writing: Assisting with writing stories, scripts, and other creative content.
- Chatbots: Powering chatbots that require a more natural and creative conversational style.
Limitations and Bias
Like all language models, Anubis-559M may generate biased or offensive content based on the data it was trained on. It is important to use the model responsibly and to be aware of its potential limitations.
How to Use
You can use this model with the transformers library in Python. Here's how to run inference:
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("RaghuCourage9605/Anubis-559M")
model = AutoModelForCausalLM.from_pretrained("RaghuCourage9605/Anubis-559M")
input_ids = tokenizer("Hello world", return_tensors="pt").input_ids
output_ids = model.generate(
input_ids,
max_new_tokens=100,
do_sample=True,
top_k=50,
top_p=0.95,
temperature=1.0
)
print(tokenizer.decode(output_ids[0]))
- Downloads last month
- -