Text Generation
Transformers
PyTorch
Safetensors
English
llama
llama-2
astronomy
astrophysics
arxiv
text-generation-inference
Instructions to use UniverseTBD/astrollama with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use UniverseTBD/astrollama with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="UniverseTBD/astrollama")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("UniverseTBD/astrollama") model = AutoModelForCausalLM.from_pretrained("UniverseTBD/astrollama") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use UniverseTBD/astrollama with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "UniverseTBD/astrollama" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "UniverseTBD/astrollama", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/UniverseTBD/astrollama
- SGLang
How to use UniverseTBD/astrollama with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "UniverseTBD/astrollama" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "UniverseTBD/astrollama", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "UniverseTBD/astrollama" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "UniverseTBD/astrollama", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use UniverseTBD/astrollama with Docker Model Runner:
docker model run hf.co/UniverseTBD/astrollama
Commit History
Update README.md d41ff0e
Add HF space a21ce55
Update emb method be83937
Upload folder using huggingface_hub 8fb7a30
Upload folder using huggingface_hub 11262b9
Upload folder using huggingface_hub 69e62a7
remove adapter stuff 3710801
Josh Nguyen commited on
Update README.md c4f4220
Update README.md 6c1920c
Update README.md f6eeb96
Update README.md a6f4193
Update README.md 3ee1192
Add model logo 4fc6d9e
Remove runs 5f1a62b
Josh Nguyen commited on
Update README.md bf67fd7
Upload folder using huggingface_hub ba541c6
Upload tensorboard/events.out.tfevents.1691575925.gadi-gpu-rsaa-0001.gadi.nci.org.au.3735927.0 with huggingface_hub 04ba8e0
Upload runs/events.out.tfevents.1691575925.gadi-gpu-rsaa-0001.gadi.nci.org.au.3735927.0 with huggingface_hub ce8e171
Upload runs/events.out.tfevents.1691575925.gadi-gpu-rsaa-0001.gadi.nci.org.au.3735927.0 with huggingface_hub 813ce5a
Upload folder using huggingface_hub 3657de9
Upload folder using huggingface_hub 4d6228a
Upload folder using huggingface_hub 8d3b22e
initial commit 5c772b0
Josh Nguyen commited on