Instructions to use ayh015/myLightningOPD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ayh015/myLightningOPD with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ayh015/myLightningOPD")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ayh015/myLightningOPD")
model = AutoModelForCausalLM.from_pretrained("ayh015/myLightningOPD", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ayh015/myLightningOPD with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ayh015/myLightningOPD"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ayh015/myLightningOPD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ayh015/myLightningOPD

SGLang

How to use ayh015/myLightningOPD with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ayh015/myLightningOPD" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ayh015/myLightningOPD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ayh015/myLightningOPD" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ayh015/myLightningOPD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ayh015/myLightningOPD with Docker Model Runner:
```
docker model run hf.co/ayh015/myLightningOPD
```

myLightningOPD / slime_plugins /rollout_buffer /README.md

ayh015

Upload folder using huggingface_hub

d588dab verified about 2 months ago

preview code

Raw

History Blame Contribute Delete

2.12 kB

Rollout Buffer

Overview

Rollout Buffer is an independent component for asynchronous agent trajectory generation, with the main function of using the LLM OpenAI Server launched by slime training to generate agent trajectories.

Workflow

slime Training Process ←─── HTTP API ───→ Rollout Buffer
        ↓                                      ↓
   LLM Server ←─────── HTTP Requests ─────── Agent Framework
        ↓                                      ↓
   Model Response ──────────────────────→ Trajectory Generation

For each different Agent task, there should be a corresponding independent Generator class, responsible for generating trajectories for that type of task. Rollout Buffer automatically reads and loads different types of Generators.

Quick Start

Basic Usage Process

Copy Template: Copy base_generator.py as a template
Modify Task Type: Change TASK_TYPE to your task name (cannot duplicate with other Generators)
Implement Core Function: Implement the run_rollout() function
Optional Customization: Rewrite five optional functions as needed

Generator files must end with _generator.py and be placed in the generator/ directory:

generator/
├── base_generator.py      # Math task implementation (default template)
└── your_task_generator.py # Your custom task

Each Generator file must define TASK_TYPE and run_rollout().

In addition, Rollout Buffer also provides some customizable functions to meet special needs of different tasks. If no custom implementation is provided, the system will use default implementations (located in slime_plugins/rollout_buffer/default_func.py).

Example Script

First, you need to follow Example: Qwen3-4B Model to configure the environment, download data and convert model checkpoints. And then run the following scripts:

cd slime_plugins/rollout_buffer
bash rollout_buffer_example.sh

# In a different terminal
python buffer.py