Instructions to use xiaomoguhzz/VisionEncoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use xiaomoguhzz/VisionEncoder with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("xiaomoguhzz/VisionEncoder", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Restructure repo into data/ + legacy/ (batch 1)
Browse filesThis view is limited to 50 files because it contains too many changes. Β See raw diff
- .gitattributes +2 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/latest +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/mp_rank_00_model_states.pt +0 -0
- {declip_siglip2 β legacy/declip_siglip2}/spatial_align/zero_to_fp32.py +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/args.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/args.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/chat_template.jinja +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/generation_config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/mp_rank_00_model_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/latest +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/model.safetensors +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/processor_config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_0.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_1.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_2.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_3.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_4.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_5.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_6.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_7.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/scheduler.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/tokenizer.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/tokenizer_config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/trainer_state.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/training_args.bin +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/zero_to_fp32.py +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/logging.jsonl +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/args.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/args.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/chat_template.jinja +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/generation_config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt +0 -0
.gitattributes
CHANGED
|
@@ -54,3 +54,5 @@ ms-swift-data/video_sft_full_v800k_sharegpt.json filter=lfs diff=lfs merge=lfs -
|
|
| 54 |
ms-swift-data/video_sft_small_10pct.json filter=lfs diff=lfs merge=lfs -text
|
| 55 |
ms-swift-data/video_sft_small_10pct_sharegpt.json filter=lfs diff=lfs merge=lfs -text
|
| 56 |
llava_video/llava_video_178k_v9_decord_good_manifest.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 54 |
ms-swift-data/video_sft_small_10pct.json filter=lfs diff=lfs merge=lfs -text
|
| 55 |
ms-swift-data/video_sft_small_10pct_sharegpt.json filter=lfs diff=lfs merge=lfs -text
|
| 56 |
llava_video/llava_video_178k_v9_decord_good_manifest.json filter=lfs diff=lfs merge=lfs -text
|
| 57 |
+
legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 58 |
+
legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/latest
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/step011088/mp_rank_00_model_states.pt
RENAMED
|
File without changes
|
{declip_siglip2 β legacy/declip_siglip2}/spatial_align/zero_to_fp32.py
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/args.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/args.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/chat_template.jinja
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/generation_config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/global_step2181/mp_rank_00_model_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/latest
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/model.safetensors
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/processor_config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_0.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_1.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_2.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_3.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_4.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_5.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_6.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/rng_state_7.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/scheduler.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/tokenizer.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/tokenizer_config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/trainer_state.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/training_args.bin
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/checkpoint-2181/zero_to_fp32.py
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_kd_pretrain/logging.jsonl
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/args.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/args.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/chat_template.jinja
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/generation_config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|