Instructions to use Henk717/airochronos-33B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Henk717/airochronos-33B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Henk717/airochronos-33B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Henk717/airochronos-33B") model = AutoModelForCausalLM.from_pretrained("Henk717/airochronos-33B") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Henk717/airochronos-33B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Henk717/airochronos-33B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Henk717/airochronos-33B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Henk717/airochronos-33B
- SGLang
How to use Henk717/airochronos-33B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Henk717/airochronos-33B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Henk717/airochronos-33B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Henk717/airochronos-33B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Henk717/airochronos-33B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Henk717/airochronos-33B with Docker Model Runner:
docker model run hf.co/Henk717/airochronos-33B
Basic mathematic operation
#3
by DjaiPyh - opened
This model seems really bad at basic math operation...
Model (from @TheBloke ) and converted in GUFF:
airochronos-33b.ggmlv3.q4_K_M.bin
Here is my last chat (using llama.cpp) :
Boris: 2+2+5+1-10 = ?
Llama: Your answer is -3
Boris: Develop
Llama: Your answer is [4, -6]
Boris: explain each step
Llama: Firstly, you need to perform the operations in parenthesis. So, (2+2) + 5 - 10 gives us -1. Next, we'll do the subtraction inside the second set of parentheses, giving us [-6]. Finally, we have two values for this expression: [4, -6], which means that there are two possible solutions (-6 and 4).
Boris: but the real answer is 0
Llama: Sorry Boris, my algorithm only checks if your solution is correct or not. If it was a multiple choice question with four options, I would be able to tell you which one is right.
Boris: -10+1-2+3+8 = ? (Explain each step, simplify and then give the final answer)
Llama: The order of operations states that exponents should be done first, followed by multiplication/division from left to right, and addition/subtraction from left to right. Since there are no exponents here, let's start with multiplication/division. We can see that -10 * 1 is just -10, while 1 / -2 is infinity since division by zero is undefined. Now, moving on to addition/subtraction, we get -10 + 1 - 2 + 3 + 8, which simplifies to -7 + 5 = -2. Therefore, the final answer is -2.