Instructions to use anwgpt/anwllama-1-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use anwgpt/anwllama-1-chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="anwgpt/anwllama-1-chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("anwgpt/anwllama-1-chat")
model = AutoModelForCausalLM.from_pretrained("anwgpt/anwllama-1-chat", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use anwgpt/anwllama-1-chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "anwgpt/anwllama-1-chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anwgpt/anwllama-1-chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/anwgpt/anwllama-1-chat

SGLang

How to use anwgpt/anwllama-1-chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "anwgpt/anwllama-1-chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anwgpt/anwllama-1-chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "anwgpt/anwllama-1-chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anwgpt/anwllama-1-chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use anwgpt/anwllama-1-chat with Docker Model Runner:
```
docker model run hf.co/anwgpt/anwllama-1-chat
```

anwllama 1

Introducing anwllama 1! A experimental version of anwgpt4 using llama architecture!

Now it uses ChatLM format!

The model is under trained

User: What is artificial intelligence?
Assistant: Yes, one can do this in-depth data.  It does not have the capability to test and optimize their research. The algorithm can be used as an array of tasks, such as data or applications, using the process of solving and testing methods.

--------------------------------------------------

User: How do neural networks work?
Assistant: Yes, some high-quality devices that use artificial intelligence can be used to solve a problem-solving process and provide a comprehensive value for their security. It is often used in technology, allowing for more efficient solutions and strategies to improve communication. Additionally, solar-based systems also helps reduce air pollution, including reducing performance, depression, and improved costs.

--------------------------------------------------

User: Who created you?
Assistant: Unitedes would be used to create an algorithm with the result of data, including the following:

1. Open the data from the user and enter the data
2. The program should then display the data in the server
3. Use the input array to ensure that the data is invalid or not in use
4. If it is done by the input input and then the user has any questions or concerns about how it can handle the input value of the data
5. Finally, the program will find the number of examples in the code for each data and the amount required to be calculated

Downloads last month: 14

Safetensors

Model size

34M params

Tensor type

F32