Text Generation
Transformers
Safetensors
qwen3
llama-factory
full
Generated from Trainer
conversational
text-generation-inference
Instructions to use ayh015/myLightningOPD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ayh015/myLightningOPD with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ayh015/myLightningOPD") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ayh015/myLightningOPD") model = AutoModelForCausalLM.from_pretrained("ayh015/myLightningOPD", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ayh015/myLightningOPD with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ayh015/myLightningOPD" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ayh015/myLightningOPD", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ayh015/myLightningOPD
- SGLang
How to use ayh015/myLightningOPD with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ayh015/myLightningOPD" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ayh015/myLightningOPD", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ayh015/myLightningOPD" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ayh015/myLightningOPD", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use ayh015/myLightningOPD with Docker Model Runner:
docker model run hf.co/ayh015/myLightningOPD
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |
| # SPDX-License-Identifier: Apache-2.0 | |
| import datetime | |
| import logging | |
| import os | |
| from slime.utils.misc import SingletonMeta | |
| try: | |
| from torch.utils.tensorboard import SummaryWriter | |
| except ImportError: | |
| SummaryWriter = None | |
| logger = logging.getLogger(__name__) | |
| class _TensorboardAdapter(metaclass=SingletonMeta): | |
| _writer = None | |
| """ | |
| # Usage example: This will return the same instance every rank | |
| # tb = _TensorboardAdapter(args) # Initialize on first call | |
| # tb.log({"Loss": 0.1}, step=1) | |
| # In other files: | |
| # from tensorboard_utils import _TensorboardAdapter | |
| # tb = _TensorboardAdapter(args) # No parameters needed to get existing instance | |
| # tb.log({"Accuracy": 0.9}, step=1) | |
| """ | |
| def __init__(self, args): | |
| assert args.use_tensorboard, f"{args.use_tensorboard=}" | |
| tb_project_name = args.tb_project_name | |
| tb_experiment_name = args.tb_experiment_name | |
| if tb_project_name is not None or os.environ.get("TENSORBOARD_DIR", None): | |
| if tb_project_name is not None and tb_experiment_name is None: | |
| tb_experiment_name = datetime.datetime.now().strftime("%Y%m%d_%H%M%S") | |
| self._initialize(tb_project_name, tb_experiment_name) | |
| else: | |
| raise ValueError("tb_project_name and tb_experiment_name, or TENSORBOARD_DIR are required") | |
| def _initialize(self, tb_project_name, tb_experiment_name): | |
| """Actual initialization logic""" | |
| # Get tensorboard directory from environment variable or use default path | |
| tensorboard_dir = os.environ.get("TENSORBOARD_DIR", f"tensorboard_log/{tb_project_name}/{tb_experiment_name}") | |
| os.makedirs(tensorboard_dir, exist_ok=True) | |
| logger.info(f"Saving tensorboard log to {tensorboard_dir}.") | |
| self._writer = SummaryWriter(tensorboard_dir) | |
| def log(self, data, step): | |
| """Log data to tensorboard | |
| Args: | |
| data (dict): Dictionary containing metric names and values | |
| step (int): Current step/epoch number | |
| """ | |
| for key in data: | |
| self._writer.add_scalar(key, data[key], step) | |
| def finish(self): | |
| """Close the tensorboard writer""" | |
| self._writer.close() | |