Text Generation
Transformers
Safetensors
qwen3
llama-factory
full
Generated from Trainer
conversational
text-generation-inference
Instructions to use ayh015/myLightningOPD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ayh015/myLightningOPD with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ayh015/myLightningOPD") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ayh015/myLightningOPD") model = AutoModelForCausalLM.from_pretrained("ayh015/myLightningOPD", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ayh015/myLightningOPD with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ayh015/myLightningOPD" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ayh015/myLightningOPD", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ayh015/myLightningOPD
- SGLang
How to use ayh015/myLightningOPD with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ayh015/myLightningOPD" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ayh015/myLightningOPD", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ayh015/myLightningOPD" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ayh015/myLightningOPD", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use ayh015/myLightningOPD with Docker Model Runner:
docker model run hf.co/ayh015/myLightningOPD
Upload folder using huggingface_hub
Browse files- slime.egg-info/PKG-INFO +39 -0
- slime.egg-info/SOURCES.txt +140 -0
- slime.egg-info/dependency_links.txt +1 -0
- slime.egg-info/requires.txt +22 -0
- slime.egg-info/top_level.txt +2 -0
slime.egg-info/PKG-INFO
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Metadata-Version: 2.4
|
| 2 |
+
Name: slime
|
| 3 |
+
Version: 0.1.0
|
| 4 |
+
Author: slime Team
|
| 5 |
+
Classifier: Programming Language :: Python :: 3.10
|
| 6 |
+
Classifier: Programming Language :: Python :: 3.11
|
| 7 |
+
Classifier: Programming Language :: Python :: 3.12
|
| 8 |
+
Classifier: Environment :: GPU :: NVIDIA CUDA
|
| 9 |
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
| 10 |
+
Classifier: Topic :: System :: Distributed Computing
|
| 11 |
+
Requires-Python: >=3.10
|
| 12 |
+
License-File: LICENSE
|
| 13 |
+
Requires-Dist: accelerate
|
| 14 |
+
Requires-Dist: blobfile
|
| 15 |
+
Requires-Dist: datasets
|
| 16 |
+
Requires-Dist: httpx[http2]
|
| 17 |
+
Requires-Dist: mcp[cli]
|
| 18 |
+
Requires-Dist: megatron-bridge @ git+https://github.com/fzyzcjy/Megatron-Bridge.git@dev_rl
|
| 19 |
+
Requires-Dist: memray
|
| 20 |
+
Requires-Dist: nvidia-modelopt[torch]>=0.37.0
|
| 21 |
+
Requires-Dist: omegaconf
|
| 22 |
+
Requires-Dist: pillow
|
| 23 |
+
Requires-Dist: pylatexenc
|
| 24 |
+
Requires-Dist: pyyaml
|
| 25 |
+
Requires-Dist: ray[default]
|
| 26 |
+
Requires-Dist: ring_flash_attn
|
| 27 |
+
Requires-Dist: sglang-router>=0.2.3
|
| 28 |
+
Requires-Dist: tensorboard
|
| 29 |
+
Requires-Dist: transformers
|
| 30 |
+
Requires-Dist: wandb
|
| 31 |
+
Requires-Dist: liger_kernel
|
| 32 |
+
Provides-Extra: fsdp
|
| 33 |
+
Requires-Dist: torch>=2.0; extra == "fsdp"
|
| 34 |
+
Dynamic: author
|
| 35 |
+
Dynamic: classifier
|
| 36 |
+
Dynamic: license-file
|
| 37 |
+
Dynamic: provides-extra
|
| 38 |
+
Dynamic: requires-dist
|
| 39 |
+
Dynamic: requires-python
|
slime.egg-info/SOURCES.txt
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
LICENSE
|
| 2 |
+
README.md
|
| 3 |
+
pyproject.toml
|
| 4 |
+
setup.py
|
| 5 |
+
slime/__init__.py
|
| 6 |
+
slime.egg-info/PKG-INFO
|
| 7 |
+
slime.egg-info/SOURCES.txt
|
| 8 |
+
slime.egg-info/dependency_links.txt
|
| 9 |
+
slime.egg-info/requires.txt
|
| 10 |
+
slime.egg-info/top_level.txt
|
| 11 |
+
slime/backends/__init__.py
|
| 12 |
+
slime/backends/fsdp_utils/__init__.py
|
| 13 |
+
slime/backends/fsdp_utils/actor.py
|
| 14 |
+
slime/backends/fsdp_utils/arguments.py
|
| 15 |
+
slime/backends/fsdp_utils/checkpoint.py
|
| 16 |
+
slime/backends/fsdp_utils/data_packing.py
|
| 17 |
+
slime/backends/fsdp_utils/lr_scheduler.py
|
| 18 |
+
slime/backends/fsdp_utils/update_weight_utils.py
|
| 19 |
+
slime/backends/fsdp_utils/kernels/__init__.py
|
| 20 |
+
slime/backends/fsdp_utils/kernels/fused_experts.py
|
| 21 |
+
slime/backends/fsdp_utils/kernels/fused_moe_triton_backward_kernels.py
|
| 22 |
+
slime/backends/fsdp_utils/models/__init__.py
|
| 23 |
+
slime/backends/fsdp_utils/models/qwen3_moe.py
|
| 24 |
+
slime/backends/fsdp_utils/models/qwen3_moe_hf.py
|
| 25 |
+
slime/backends/megatron_utils/__init__.py
|
| 26 |
+
slime/backends/megatron_utils/actor.py
|
| 27 |
+
slime/backends/megatron_utils/arguments.py
|
| 28 |
+
slime/backends/megatron_utils/checkpoint.py
|
| 29 |
+
slime/backends/megatron_utils/ci_utils.py
|
| 30 |
+
slime/backends/megatron_utils/cp_utils.py
|
| 31 |
+
slime/backends/megatron_utils/data.py
|
| 32 |
+
slime/backends/megatron_utils/initialize.py
|
| 33 |
+
slime/backends/megatron_utils/loss.py
|
| 34 |
+
slime/backends/megatron_utils/misc_utils.py
|
| 35 |
+
slime/backends/megatron_utils/model.py
|
| 36 |
+
slime/backends/megatron_utils/model_provider.py
|
| 37 |
+
slime/backends/megatron_utils/sglang.py
|
| 38 |
+
slime/backends/megatron_utils/megatron_to_hf/__init__.py
|
| 39 |
+
slime/backends/megatron_utils/megatron_to_hf/deepseekv3.py
|
| 40 |
+
slime/backends/megatron_utils/megatron_to_hf/glm4.py
|
| 41 |
+
slime/backends/megatron_utils/megatron_to_hf/glm4moe.py
|
| 42 |
+
slime/backends/megatron_utils/megatron_to_hf/llama.py
|
| 43 |
+
slime/backends/megatron_utils/megatron_to_hf/mimo.py
|
| 44 |
+
slime/backends/megatron_utils/megatron_to_hf/qwen2.py
|
| 45 |
+
slime/backends/megatron_utils/megatron_to_hf/qwen3_next.py
|
| 46 |
+
slime/backends/megatron_utils/megatron_to_hf/qwen3moe.py
|
| 47 |
+
slime/backends/megatron_utils/megatron_to_hf/processors/__init__.py
|
| 48 |
+
slime/backends/megatron_utils/megatron_to_hf/processors/padding_remover.py
|
| 49 |
+
slime/backends/megatron_utils/megatron_to_hf/processors/quantizer.py
|
| 50 |
+
slime/backends/megatron_utils/update_weight/__init__.py
|
| 51 |
+
slime/backends/megatron_utils/update_weight/common.py
|
| 52 |
+
slime/backends/megatron_utils/update_weight/hf_weight_iterator_base.py
|
| 53 |
+
slime/backends/megatron_utils/update_weight/hf_weight_iterator_bridge.py
|
| 54 |
+
slime/backends/megatron_utils/update_weight/hf_weight_iterator_direct.py
|
| 55 |
+
slime/backends/megatron_utils/update_weight/update_weight_from_distributed.py
|
| 56 |
+
slime/backends/megatron_utils/update_weight/update_weight_from_tensor.py
|
| 57 |
+
slime/backends/sglang_utils/__init__.py
|
| 58 |
+
slime/backends/sglang_utils/arguments.py
|
| 59 |
+
slime/backends/sglang_utils/sglang_engine.py
|
| 60 |
+
slime/ray/__init__.py
|
| 61 |
+
slime/ray/actor_group.py
|
| 62 |
+
slime/ray/placement_group.py
|
| 63 |
+
slime/ray/ray_actor.py
|
| 64 |
+
slime/ray/rollout.py
|
| 65 |
+
slime/ray/train_actor.py
|
| 66 |
+
slime/ray/utils.py
|
| 67 |
+
slime/rollout/__init__.py
|
| 68 |
+
slime/rollout/base_types.py
|
| 69 |
+
slime/rollout/data_source.py
|
| 70 |
+
slime/rollout/on_policy_distillation.py
|
| 71 |
+
slime/rollout/sglang_rollout.py
|
| 72 |
+
slime/rollout/sleep_rollout.py
|
| 73 |
+
slime/rollout/filter_hub/__init__.py
|
| 74 |
+
slime/rollout/filter_hub/base_types.py
|
| 75 |
+
slime/rollout/filter_hub/dynamic_sampling_filters.py
|
| 76 |
+
slime/rollout/rm_hub/__init__.py
|
| 77 |
+
slime/rollout/rm_hub/deepscaler.py
|
| 78 |
+
slime/rollout/rm_hub/f1.py
|
| 79 |
+
slime/rollout/rm_hub/gpqa.py
|
| 80 |
+
slime/rollout/rm_hub/ifbench.py
|
| 81 |
+
slime/rollout/rm_hub/math_dapo_utils.py
|
| 82 |
+
slime/rollout/rm_hub/math_utils.py
|
| 83 |
+
slime/router/__init__.py
|
| 84 |
+
slime/router/router.py
|
| 85 |
+
slime/router/middleware_hub/__init__.py
|
| 86 |
+
slime/router/middleware_hub/radix_tree.py
|
| 87 |
+
slime/router/middleware_hub/radix_tree_middleware.py
|
| 88 |
+
slime/utils/__init__.py
|
| 89 |
+
slime/utils/arguments.py
|
| 90 |
+
slime/utils/async_utils.py
|
| 91 |
+
slime/utils/context_utils.py
|
| 92 |
+
slime/utils/data.py
|
| 93 |
+
slime/utils/distributed_utils.py
|
| 94 |
+
slime/utils/eval_config.py
|
| 95 |
+
slime/utils/flops_utils.py
|
| 96 |
+
slime/utils/fp8_kernel.py
|
| 97 |
+
slime/utils/health_monitor.py
|
| 98 |
+
slime/utils/http_utils.py
|
| 99 |
+
slime/utils/iter_utils.py
|
| 100 |
+
slime/utils/logging_utils.py
|
| 101 |
+
slime/utils/mask_utils.py
|
| 102 |
+
slime/utils/megatron_bridge_utils.py
|
| 103 |
+
slime/utils/memory_utils.py
|
| 104 |
+
slime/utils/metric_checker.py
|
| 105 |
+
slime/utils/metric_utils.py
|
| 106 |
+
slime/utils/misc.py
|
| 107 |
+
slime/utils/ppo_utils.py
|
| 108 |
+
slime/utils/processing_utils.py
|
| 109 |
+
slime/utils/profile_utils.py
|
| 110 |
+
slime/utils/ray_utils.py
|
| 111 |
+
slime/utils/reloadable_process_group.py
|
| 112 |
+
slime/utils/rocm_checkpoint_writer.py
|
| 113 |
+
slime/utils/routing_replay.py
|
| 114 |
+
slime/utils/seqlen_balancing.py
|
| 115 |
+
slime/utils/tensor_backper.py
|
| 116 |
+
slime/utils/tensorboard_utils.py
|
| 117 |
+
slime/utils/timer.py
|
| 118 |
+
slime/utils/tracking_utils.py
|
| 119 |
+
slime/utils/train_dump_utils.py
|
| 120 |
+
slime/utils/train_metric_utils.py
|
| 121 |
+
slime/utils/typer_utils.py
|
| 122 |
+
slime/utils/types.py
|
| 123 |
+
slime/utils/wandb_utils.py
|
| 124 |
+
slime/utils/debug_utils/__init__.py
|
| 125 |
+
slime/utils/debug_utils/display_debug_rollout_data.py
|
| 126 |
+
slime/utils/debug_utils/replay_reward_fn.py
|
| 127 |
+
slime/utils/debug_utils/send_to_sglang.py
|
| 128 |
+
slime/utils/external_utils/__init__.py
|
| 129 |
+
slime/utils/external_utils/command_utils.py
|
| 130 |
+
slime_plugins/__init__.py
|
| 131 |
+
slime_plugins/mbridge/__init__.py
|
| 132 |
+
slime_plugins/mbridge/glm4.py
|
| 133 |
+
slime_plugins/mbridge/glm4moe.py
|
| 134 |
+
slime_plugins/mbridge/mimo.py
|
| 135 |
+
slime_plugins/mbridge/qwen3_next.py
|
| 136 |
+
slime_plugins/megatron_bridge/__init__.py
|
| 137 |
+
slime_plugins/models/__init__.py
|
| 138 |
+
slime_plugins/models/glm4.py
|
| 139 |
+
slime_plugins/models/hf_attention.py
|
| 140 |
+
slime_plugins/models/qwen3_next.py
|
slime.egg-info/dependency_links.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
|
slime.egg-info/requires.txt
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
accelerate
|
| 2 |
+
blobfile
|
| 3 |
+
datasets
|
| 4 |
+
httpx[http2]
|
| 5 |
+
mcp[cli]
|
| 6 |
+
megatron-bridge @ git+https://github.com/fzyzcjy/Megatron-Bridge.git@dev_rl
|
| 7 |
+
memray
|
| 8 |
+
nvidia-modelopt[torch]>=0.37.0
|
| 9 |
+
omegaconf
|
| 10 |
+
pillow
|
| 11 |
+
pylatexenc
|
| 12 |
+
pyyaml
|
| 13 |
+
ray[default]
|
| 14 |
+
ring_flash_attn
|
| 15 |
+
sglang-router>=0.2.3
|
| 16 |
+
tensorboard
|
| 17 |
+
transformers
|
| 18 |
+
wandb
|
| 19 |
+
liger_kernel
|
| 20 |
+
|
| 21 |
+
[fsdp]
|
| 22 |
+
torch>=2.0
|
slime.egg-info/top_level.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
slime
|
| 2 |
+
slime_plugins
|