Video-Text-to-Text
Transformers
Safetensors
English
qwen2
text-generation
Action
Video
MQA
multimodal
MLLMs
LLaVAction
text-generation-inference
Instructions to use MLAdaptiveIntelligence/LLaVAction-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MLAdaptiveIntelligence/LLaVAction-0.5B with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("MLAdaptiveIntelligence/LLaVAction-0.5B") model = AutoModelForCausalLM.from_pretrained("MLAdaptiveIntelligence/LLaVAction-0.5B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -81,7 +81,7 @@ video,frame_time,video_time = load_video(video_path, max_frames_num, 1, force_sa
|
|
| 81 |
video = image_processor.preprocess(video, return_tensors="pt")["pixel_values"].cuda().half()
|
| 82 |
video = [video]
|
| 83 |
conv_template = "qwen_1_5" # Make sure you use correct chat template for different models
|
| 84 |
-
|
| 85 |
perspective_prompt = "You are seeing this video from egocentric view and you are the person. Your hands are sometimes interacting with objects. What action are you doing?"
|
| 86 |
task_prompt = "Describe in details what you see from the video frames."
|
| 87 |
question = DEFAULT_IMAGE_TOKEN + f"\n{time_instruction}\n{perspective_prompt} {task_prompt}"
|
|
|
|
| 81 |
video = image_processor.preprocess(video, return_tensors="pt")["pixel_values"].cuda().half()
|
| 82 |
video = [video]
|
| 83 |
conv_template = "qwen_1_5" # Make sure you use correct chat template for different models
|
| 84 |
+
time_instruction = f"The video lasts for {video_time:.2f} seconds, and {len(video[0])} frames are uniformly sampled from it. "
|
| 85 |
perspective_prompt = "You are seeing this video from egocentric view and you are the person. Your hands are sometimes interacting with objects. What action are you doing?"
|
| 86 |
task_prompt = "Describe in details what you see from the video frames."
|
| 87 |
question = DEFAULT_IMAGE_TOKEN + f"\n{time_instruction}\n{perspective_prompt} {task_prompt}"
|