Instructions to use kaedefuto/chat_bot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use kaedefuto/chat_bot with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="kaedefuto/chat_bot")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("kaedefuto/chat_bot") model = AutoModelForCausalLM.from_pretrained("kaedefuto/chat_bot") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use kaedefuto/chat_bot with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "kaedefuto/chat_bot" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kaedefuto/chat_bot", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/kaedefuto/chat_bot
- SGLang
How to use kaedefuto/chat_bot with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "kaedefuto/chat_bot" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kaedefuto/chat_bot", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "kaedefuto/chat_bot" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kaedefuto/chat_bot", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use kaedefuto/chat_bot with Docker Model Runner:
docker model run hf.co/kaedefuto/chat_bot
| { | |
| "best_metric": null, | |
| "best_model_checkpoint": null, | |
| "epoch": 3.0, | |
| "global_step": 13038, | |
| "is_hyper_param_search": false, | |
| "is_local_process_zero": true, | |
| "is_world_process_zero": true, | |
| "log_history": [ | |
| { | |
| "epoch": 0.12, | |
| "learning_rate": 4.808252799509128e-05, | |
| "loss": 3.9076, | |
| "step": 500 | |
| }, | |
| { | |
| "epoch": 0.23, | |
| "learning_rate": 4.6165055990182545e-05, | |
| "loss": 3.6869, | |
| "step": 1000 | |
| }, | |
| { | |
| "epoch": 0.35, | |
| "learning_rate": 4.424758398527381e-05, | |
| "loss": 3.6372, | |
| "step": 1500 | |
| }, | |
| { | |
| "epoch": 0.46, | |
| "learning_rate": 4.233011198036509e-05, | |
| "loss": 3.612, | |
| "step": 2000 | |
| }, | |
| { | |
| "epoch": 0.58, | |
| "learning_rate": 4.041263997545636e-05, | |
| "loss": 3.5812, | |
| "step": 2500 | |
| }, | |
| { | |
| "epoch": 0.69, | |
| "learning_rate": 3.849516797054763e-05, | |
| "loss": 3.5516, | |
| "step": 3000 | |
| }, | |
| { | |
| "epoch": 0.81, | |
| "learning_rate": 3.6577695965638905e-05, | |
| "loss": 3.5391, | |
| "step": 3500 | |
| }, | |
| { | |
| "epoch": 0.92, | |
| "learning_rate": 3.466022396073017e-05, | |
| "loss": 3.5271, | |
| "step": 4000 | |
| }, | |
| { | |
| "epoch": 1.04, | |
| "learning_rate": 3.274275195582145e-05, | |
| "loss": 3.4532, | |
| "step": 4500 | |
| }, | |
| { | |
| "epoch": 1.15, | |
| "learning_rate": 3.0825279950912716e-05, | |
| "loss": 3.2922, | |
| "step": 5000 | |
| }, | |
| { | |
| "epoch": 1.27, | |
| "learning_rate": 2.890780794600399e-05, | |
| "loss": 3.3032, | |
| "step": 5500 | |
| }, | |
| { | |
| "epoch": 1.38, | |
| "learning_rate": 2.6990335941095262e-05, | |
| "loss": 3.3062, | |
| "step": 6000 | |
| }, | |
| { | |
| "epoch": 1.5, | |
| "learning_rate": 2.507286393618653e-05, | |
| "loss": 3.319, | |
| "step": 6500 | |
| }, | |
| { | |
| "epoch": 1.61, | |
| "learning_rate": 2.3155391931277805e-05, | |
| "loss": 3.2838, | |
| "step": 7000 | |
| }, | |
| { | |
| "epoch": 1.73, | |
| "learning_rate": 2.1237919926369076e-05, | |
| "loss": 3.2729, | |
| "step": 7500 | |
| }, | |
| { | |
| "epoch": 1.84, | |
| "learning_rate": 1.9320447921460347e-05, | |
| "loss": 3.2908, | |
| "step": 8000 | |
| }, | |
| { | |
| "epoch": 1.96, | |
| "learning_rate": 1.740297591655162e-05, | |
| "loss": 3.2716, | |
| "step": 8500 | |
| }, | |
| { | |
| "epoch": 2.07, | |
| "learning_rate": 1.548550391164289e-05, | |
| "loss": 3.1832, | |
| "step": 9000 | |
| }, | |
| { | |
| "epoch": 2.19, | |
| "learning_rate": 1.3568031906734163e-05, | |
| "loss": 3.13, | |
| "step": 9500 | |
| }, | |
| { | |
| "epoch": 2.3, | |
| "learning_rate": 1.1650559901825433e-05, | |
| "loss": 3.1337, | |
| "step": 10000 | |
| }, | |
| { | |
| "epoch": 2.42, | |
| "learning_rate": 9.733087896916706e-06, | |
| "loss": 3.1165, | |
| "step": 10500 | |
| }, | |
| { | |
| "epoch": 2.53, | |
| "learning_rate": 7.815615892007977e-06, | |
| "loss": 3.1283, | |
| "step": 11000 | |
| }, | |
| { | |
| "epoch": 2.65, | |
| "learning_rate": 5.8981438870992485e-06, | |
| "loss": 3.1381, | |
| "step": 11500 | |
| }, | |
| { | |
| "epoch": 2.76, | |
| "learning_rate": 3.980671882190521e-06, | |
| "loss": 3.1443, | |
| "step": 12000 | |
| }, | |
| { | |
| "epoch": 2.88, | |
| "learning_rate": 2.0631998772817916e-06, | |
| "loss": 3.1399, | |
| "step": 12500 | |
| }, | |
| { | |
| "epoch": 2.99, | |
| "learning_rate": 1.4572787237306336e-07, | |
| "loss": 3.1344, | |
| "step": 13000 | |
| }, | |
| { | |
| "epoch": 3.0, | |
| "step": 13038, | |
| "total_flos": 2.6925692092416e+16, | |
| "train_runtime": 5223.9033, | |
| "train_samples_per_second": 2.496 | |
| } | |
| ], | |
| "max_steps": 13038, | |
| "num_train_epochs": 3, | |
| "total_flos": 2.6925692092416e+16, | |
| "trial_name": null, | |
| "trial_params": null | |
| } | |