Visual Question Answering
Transformers
Safetensors
English
videollama3_qwen2
text-generation
multi-modal
large-language-model
video-language-model
custom_code
Instructions to use DAMO-NLP-SG/VideoLLaMA3-7B-Image with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DAMO-NLP-SG/VideoLLaMA3-7B-Image with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("visual-question-answering", model="DAMO-NLP-SG/VideoLLaMA3-7B-Image", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("DAMO-NLP-SG/VideoLLaMA3-7B-Image", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
something wrong in caling ffmpeg to extract frames
#4
by menglan - opened
hi i visualize the frame extracted from processing_videollama3.py in load_video func,
out, _ = ffmpeg.run(stream, capture_stdout=True, quiet=not verbose)
frames_xxx = np.frombuffer(out, np.uint8).reshape([-1, new_h, new_w, 3])
for ixx, xframe in enumerate(frames_xxx, 1):
cv2.imwrite("rgb_{}.jpg".format(str(ixx).zfill(3)), xframe)
cv2.imwrite("bgr_{}.jpg".format(str(ixx).zfill(3)), cv2.cvtColor(xframe, cv2.COLOR_RGB2BGR))
import pdb; pdb.set_trace()
frames = np.frombuffer(out, np.uint8).reshape([-1, new_h, new_w, 3]).transpose([0, 3, 1, 2])
the saved image is abnormal, like this
is it a bug?
