Instructions to use xiaomoguhzz/VisionEncoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use xiaomoguhzz/VisionEncoder with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("xiaomoguhzz/VisionEncoder", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Restructure repo into data/ + legacy/ (batch 2)
Browse filesThis view is limited to 50 files because it contains too many changes. Β See raw diff
- .gitattributes +11 -0
- {llava_video β data/llava_video}/llava_video_178k_v9_decord_good_manifest.json +0 -0
- {ms-swift-data β data/ms-swift-data}/image_sft_full_v800k.json +0 -0
- {ms-swift-data β data/ms-swift-data}/image_sft_full_v800k_sharegpt.json +0 -0
- {ms-swift-data β data/ms-swift-data}/image_sft_small_10pct.json +0 -0
- {ms-swift-data β data/ms-swift-data}/image_sft_small_10pct_sharegpt.json +0 -0
- {ms-swift-data β data/ms-swift-data}/sft_mixed_config_full_v800k.yaml +0 -0
- {ms-swift-data β data/ms-swift-data}/sft_mixed_config_small_10pct.yaml +0 -0
- {ms-swift-data β data/ms-swift-data}/video_sft_full_v800k.json +0 -0
- {ms-swift-data β data/ms-swift-data}/video_sft_full_v800k_sharegpt.json +0 -0
- {ms-swift-data β data/ms-swift-data}/video_sft_small_10pct.json +0 -0
- {ms-swift-data β data/ms-swift-data}/video_sft_small_10pct_sharegpt.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/generation_config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/mp_rank_00_model_states.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/latest +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/model-00001-of-00002.safetensors +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/model-00002-of-00002.safetensors +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/model.safetensors.index.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/processor_config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_0.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_1.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_2.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_3.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_4.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_5.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_6.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_7.pth +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/scheduler.pt +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/tokenizer.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/tokenizer_config.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/trainer_state.json +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/training_args.bin +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/zero_to_fp32.py +0 -0
- {kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/logging.jsonl +0 -0
- {self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/args.json +0 -0
- {self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/chat_template.jinja +0 -0
- {self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/config.json +0 -0
- {self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/generation_config.json +0 -0
- {self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt +0 -0
- {self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt +0 -0
- {self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt +0 -0
.gitattributes
CHANGED
|
@@ -56,3 +56,14 @@ ms-swift-data/video_sft_small_10pct_sharegpt.json filter=lfs diff=lfs merge=lfs
|
|
| 56 |
llava_video/llava_video_178k_v9_decord_good_manifest.json filter=lfs diff=lfs merge=lfs -text
|
| 57 |
legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 58 |
legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
llava_video/llava_video_178k_v9_decord_good_manifest.json filter=lfs diff=lfs merge=lfs -text
|
| 57 |
legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 58 |
legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 59 |
+
legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 60 |
+
data/llava_video/llava_video_178k_v9_decord_good_manifest.json filter=lfs diff=lfs merge=lfs -text
|
| 61 |
+
data/ms-swift-data/image_sft_full_v800k.json filter=lfs diff=lfs merge=lfs -text
|
| 62 |
+
data/ms-swift-data/image_sft_full_v800k_sharegpt.json filter=lfs diff=lfs merge=lfs -text
|
| 63 |
+
data/ms-swift-data/image_sft_small_10pct.json filter=lfs diff=lfs merge=lfs -text
|
| 64 |
+
data/ms-swift-data/image_sft_small_10pct_sharegpt.json filter=lfs diff=lfs merge=lfs -text
|
| 65 |
+
data/ms-swift-data/video_sft_full_v800k.json filter=lfs diff=lfs merge=lfs -text
|
| 66 |
+
data/ms-swift-data/video_sft_full_v800k_sharegpt.json filter=lfs diff=lfs merge=lfs -text
|
| 67 |
+
data/ms-swift-data/video_sft_small_10pct.json filter=lfs diff=lfs merge=lfs -text
|
| 68 |
+
data/ms-swift-data/video_sft_small_10pct_sharegpt.json filter=lfs diff=lfs merge=lfs -text
|
| 69 |
+
legacy/self_refine/qwen3vl_2b_10pct/checkpoint-1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
{llava_video β data/llava_video}/llava_video_178k_v9_decord_good_manifest.json
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/image_sft_full_v800k.json
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/image_sft_full_v800k_sharegpt.json
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/image_sft_small_10pct.json
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/image_sft_small_10pct_sharegpt.json
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/sft_mixed_config_full_v800k.yaml
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/sft_mixed_config_small_10pct.yaml
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/video_sft_full_v800k.json
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/video_sft_full_v800k_sharegpt.json
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/video_sft_small_10pct.json
RENAMED
|
File without changes
|
{ms-swift-data β data/ms-swift-data}/video_sft_small_10pct_sharegpt.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/generation_config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/mp_rank_00_model_states.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/latest
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/model-00001-of-00002.safetensors
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/model-00002-of-00002.safetensors
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/model.safetensors.index.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/processor_config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_0.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_1.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_2.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_3.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_4.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_5.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_6.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/rng_state_7.pth
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/scheduler.pt
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/tokenizer.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/tokenizer_config.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/trainer_state.json
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/training_args.bin
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/zero_to_fp32.py
RENAMED
|
File without changes
|
{kd_mllm β legacy/kd_mllm}/s2_siglip2_qwen3_4b_10pct/logging.jsonl
RENAMED
|
File without changes
|
{self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/args.json
RENAMED
|
File without changes
|
{self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/chat_template.jinja
RENAMED
|
File without changes
|
{self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/config.json
RENAMED
|
File without changes
|
{self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/generation_config.json
RENAMED
|
File without changes
|
{self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|
{self_refine β legacy/self_refine}/qwen3vl_2b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
RENAMED
|
File without changes
|