Instructions to use ayh015/myLightningOPD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ayh015/myLightningOPD with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ayh015/myLightningOPD")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ayh015/myLightningOPD")
model = AutoModelForCausalLM.from_pretrained("ayh015/myLightningOPD", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ayh015/myLightningOPD with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ayh015/myLightningOPD"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ayh015/myLightningOPD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ayh015/myLightningOPD

SGLang

How to use ayh015/myLightningOPD with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ayh015/myLightningOPD" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ayh015/myLightningOPD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ayh015/myLightningOPD" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ayh015/myLightningOPD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ayh015/myLightningOPD with Docker Model Runner:
```
docker model run hf.co/ayh015/myLightningOPD
```

myLightningOPD / slime /backends /fsdp_utils /__init__.py

ayh015

Upload folder using huggingface_hub

6011e08 verified about 2 months ago

Raw

History Blame Contribute Delete

1.15 kB

	# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
	# SPDX-License-Identifier: Apache-2.0

	import logging


	try:
	_TORCH_MEMORY_SAVER_AVAILABLE = True
	except ImportError:
	logging.warning("torch_memory_saver is not installed, refer to : https://github.com/fzyzcjy/torch_memory_saver")
	_TORCH_MEMORY_SAVER_AVAILABLE = False

	try:
	_FSDP_AVAILABLE = True
	except ImportError as e:
	logging.warning(f"FSDP backend dependencies not available: {e}")
	_FSDP_AVAILABLE = False

	if _FSDP_AVAILABLE:
	from .actor import FSDPTrainRayActor
	from .arguments import load_fsdp_args
	else:

	def _raise_import_error(args, *kwargs):
	raise ImportError(
	"FSDP backend is not available. "
	"Please ensure PyTorch with FSDP2 support is installed. "
	"For installation instructions, refer to: https://pytorch.org/docs/stable/distributed.fsdp.fully_shard.html"
	)

	FSDPTrainRayActor = _raise_import_error
	load_fsdp_args = _raise_import_error

	__all__ = ["load_fsdp_args", "FSDPTrainRayActor"]

	logging.getLogger().setLevel(logging.WARNING)