Cynaptics/persona-chat
Viewer • Updated • 20k • 397 • 3
How to use DarkNeuronAI/darkneuron-chat-v1.1 with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="DarkNeuronAI/darkneuron-chat-v1.1") # Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("DarkNeuronAI/darkneuron-chat-v1.1")
model = AutoModelForCausalLM.from_pretrained("DarkNeuronAI/darkneuron-chat-v1.1")How to use DarkNeuronAI/darkneuron-chat-v1.1 with vLLM:
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DarkNeuronAI/darkneuron-chat-v1.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "DarkNeuronAI/darkneuron-chat-v1.1",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker model run hf.co/DarkNeuronAI/darkneuron-chat-v1.1
How to use DarkNeuronAI/darkneuron-chat-v1.1 with SGLang:
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
--model-path "DarkNeuronAI/darkneuron-chat-v1.1" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "DarkNeuronAI/darkneuron-chat-v1.1",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "DarkNeuronAI/darkneuron-chat-v1.1" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "DarkNeuronAI/darkneuron-chat-v1.1",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'How to use DarkNeuronAI/darkneuron-chat-v1.1 with Docker Model Runner:
docker model run hf.co/DarkNeuronAI/darkneuron-chat-v1.1
DarkNeuron-Chat v1.1 is a chatbot designed for basic, friendly conversations. It provides clear and concise responses and is suitable for general use.
Install the latest version of Transformers:
!pip install --upgrade transformers torch
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
import torch, gc
# Load tokenizer and model
model_name = "DarkNeuron-AI/darkneuron-chat-v1.1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Use GPU if available
device = 0 if torch.cuda.is_available() else -1
# Create chatbot pipeline
chatbot = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
device=device,
return_full_text=False
)
# Optional: Free GPU memory
gc.collect()
torch.cuda.empty_cache()
# Interactive chat loop
print("Chatbot ready! Type 'exit' or 'quit' to stop.\n")
while True:
user_input = input("User: ")
if user_input.lower() in ["exit", "quit"]:
print("Chat ended.")
break
prompt = f"User: {user_input}\nBot:"
response = chatbot(
prompt,
max_length=100,
do_sample=True,
temperature=0.7,
top_p=0.9,
num_return_sequences=1
)
print(response[0]["generated_text"])
Base model
distilbert/distilgpt2
docker model run hf.co/DarkNeuronAI/darkneuron-chat-v1.1