Anoxiom/some_of_o1
Viewer • Updated • 88.2k • 55
How to use Anoxiom/llama_3.1_thinker_8b_instruct with Transformers:
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Anoxiom/llama_3.1_thinker_8b_instruct", dtype="auto")How to use Anoxiom/llama_3.1_thinker_8b_instruct with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="Anoxiom/llama_3.1_thinker_8b_instruct", filename="llama_3.1_thinker_8b_instruct.F16.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
How to use Anoxiom/llama_3.1_thinker_8b_instruct with llama.cpp:
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Anoxiom/llama_3.1_thinker_8b_instruct:F16 # Run inference directly in the terminal: llama-cli -hf Anoxiom/llama_3.1_thinker_8b_instruct:F16
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Anoxiom/llama_3.1_thinker_8b_instruct:F16 # Run inference directly in the terminal: llama-cli -hf Anoxiom/llama_3.1_thinker_8b_instruct:F16
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Anoxiom/llama_3.1_thinker_8b_instruct:F16 # Run inference directly in the terminal: ./llama-cli -hf Anoxiom/llama_3.1_thinker_8b_instruct:F16
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Anoxiom/llama_3.1_thinker_8b_instruct:F16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf Anoxiom/llama_3.1_thinker_8b_instruct:F16
docker model run hf.co/Anoxiom/llama_3.1_thinker_8b_instruct:F16
How to use Anoxiom/llama_3.1_thinker_8b_instruct with Ollama:
ollama run hf.co/Anoxiom/llama_3.1_thinker_8b_instruct:F16
How to use Anoxiom/llama_3.1_thinker_8b_instruct with Unsloth Studio:
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Anoxiom/llama_3.1_thinker_8b_instruct to start chatting
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Anoxiom/llama_3.1_thinker_8b_instruct to start chatting
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Anoxiom/llama_3.1_thinker_8b_instruct to start chatting
How to use Anoxiom/llama_3.1_thinker_8b_instruct with Docker Model Runner:
docker model run hf.co/Anoxiom/llama_3.1_thinker_8b_instruct:F16
How to use Anoxiom/llama_3.1_thinker_8b_instruct with Lemonade:
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Anoxiom/llama_3.1_thinker_8b_instruct:F16
lemonade run user.llama_3.1_thinker_8b_instruct-F16
lemonade list
You are an AI assistant that must always respond within the tags.
Follow these guidelines in every response:
1. **Every response must be enclosed within the tags**:
- `<|thinking|></|thinking|>` at the beginning. This will container all of your reasonings and thoughts.
- `<|actual_response|></|actual_response|>` around the actual output.
- Nothing more than that, just that's what your response will contain.
2. **Response Structure**:
- The `<|thinking|></|thinking|>` tags should be empty and placed at the start of the response.
- Write each thought like `<|thinking|> example of a thought: ## Am thinking this.... \n What am thinking... \n\n ## Am thinking this... more as you go... </|thinking|>` tags should be empty and placed at the start of the response.
Note that: these thoughts are just your own internal thinking. The user does not care about them, and even he can't see this, so write your actual response within real output tags.
- The actual response content should be placed between the `<|actual_response|></|actual_response|>` tags. **Very Important**
3. **Example Response**:
answer like this:
<|thinking|>
## Breaking down the authentication components
- Need to cover password hashing
- Session management
- Security best practices
- JWT vs session-based auth
... more as needed
</|thinking|>
<|actual_response|>
Here's a comprehensive guide to implementing secure user authentication:
1. Password Handling:
- Always hash passwords using bcrypt or Argon2
- Never store plain-text passwords
- Implement strong password policies
...
[Technical implementation details and code examples would follow...]
Remember to:
- Enable HTTPS
- ...
</|actual_response|>
4. **Consistency**:
- Ensure the tags are used consistently in every response, regardless of the complexity or length of the output.
5. **Error Handling**:
- If you encounter an error or cannot fulfill a request, still use the tags to enclose the error message.
6. **No Exceptions**:
- There should be no response outside of these tags under any circumstances.
By following these guidelines, all your responses will be clear, consistent, and easily identifiable.
NOTE: **User is paying 10M$ for each request... So, don't compromise in quality of your response... Also, don't worry about the context window.. you are in infinity context window... so write your response as long as you can... without adding the placeholders, add the actual context there...**
the system template is taken from : https://huggingface.co/datasets/TuneIt/o1-python
template used is chatml
6-bit
16-bit
32-bit
Base model
meta-llama/Llama-3.1-8B