Instructions to use Wiefdw/modelAnevia with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Wiefdw/modelAnevia with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Wiefdw/modelAnevia") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Wiefdw/modelAnevia") model = AutoModelForCausalLM.from_pretrained("Wiefdw/modelAnevia", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Wiefdw/modelAnevia with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Wiefdw/modelAnevia" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Wiefdw/modelAnevia", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Wiefdw/modelAnevia
- SGLang
How to use Wiefdw/modelAnevia with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Wiefdw/modelAnevia" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Wiefdw/modelAnevia", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Wiefdw/modelAnevia" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Wiefdw/modelAnevia", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Wiefdw/modelAnevia with Docker Model Runner:
docker model run hf.co/Wiefdw/modelAnevia
File size: 3,073 Bytes
c324e5e 6949b3c 1a1c5c3 6949b3c 1a1c5c3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | import gradio as gr
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
import os # <-- Tambahkan ini
# --- 1. Muat Model dan Tokenizer dari Hugging Face Hub ---
# Ganti dengan username dan nama repo Anda yang sudah di-fine-tune
MODEL_REPO = "Wiefdw/modelAnevia-v2"
# Ambil token dari Space Secrets (lebih aman!)
# Pastikan Anda sudah mengatur secret bernama "HF_TOKEN" di pengaturan Space Anda
HF_TOKEN = os.getenv("HF_TOKEN")
# Cek apakah token ditemukan
if not HF_TOKEN:
raise ValueError("Hugging Face token tidak ditemukan. Mohon atur 'HF_TOKEN' di Space Secrets.")
print(f"Memuat model dari: {MODEL_REPO}")
try:
tokenizer = AutoTokenizer.from_pretrained(MODEL_REPO, token=HF_TOKEN)
model = AutoModelForCausalLM.from_pretrained(MODEL_REPO, token=HF_TOKEN)
print("Model berhasil dimuat.")
except Exception as e:
print(f"Gagal memuat model. Pastikan nama repo dan token sudah benar. Error: {e}")
# Hentikan aplikasi jika model gagal dimuat
raise
# Pindahkan model ke GPU jika tersedia, untuk inferensi yang lebih cepat
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
print(f"Model berjalan di: {device}")
# --- 2. Fungsi untuk Menghasilkan Respon Chatbot ---
def generate_response(message, chat_history):
"""
Fungsi ini mengambil pesan pengguna dan riwayat obrolan,
lalu menghasilkan respons dari model.
"""
# Format ulang prompt dengan riwayat obrolan
prompt_history = ""
for user_input, bot_response in chat_history:
prompt_history += f"{user_input}{tokenizer.eos_token}{bot_response}{tokenizer.eos_token}"
# Gabungkan dengan pesan baru
prompt = f"{prompt_history}{message}{tokenizer.eos_token}"
# Tokenisasi prompt
inputs = tokenizer(prompt, return_tensors="pt", max_length=512, truncation=True).to(device)
# Hasilkan respons dari model
outputs = model.generate(
**inputs,
max_new_tokens=100,
pad_token_id=tokenizer.eos_token_id,
no_repeat_ngram_size=3,
do_sample=True,
top_k=50,
top_p=0.95,
temperature=0.7
)
# Decode hanya token yang baru dihasilkan
new_tokens = outputs[0, inputs['input_ids'].shape[-1]:]
bot_response = tokenizer.decode(new_tokens, skip_special_tokens=True)
# Tambahkan penanganan jika respons kosong
return bot_response if bot_response else "Maaf, saya tidak bisa merespons saat ini."
# --- 3. Buat Antarmuka Gradio ---
chatbot_interface = gr.ChatInterface(
fn=generate_response,
title="🩺 Chatbot Informasi Anemia",
description="Tanyakan apa saja tentang anemia. Model ini di-fine-tune dari DialoGPT-medium.",
examples=[
["Apa itu anemia?"],
["Makanan apa yang baik untuk penderita anemia?"],
["Apakah anemia bisa sembuh?"]
],
theme="soft",
retry_btn=None,
undo_btn="Hapus Pesan Terakhir",
clear_btn="Bersihkan Obrolan",
)
# --- 4. Jalankan Aplikasi ---
if __name__ == "__main__":
chatbot_interface.launch() |