Article
Small Yet Mighty: Improve Accuracy In Multimodal Search and Visual Document Retrieval with Llama Nemotron RAG Models
nvidia
• • 31
Jose Dimas Sandoval Duran
joseEs
AI & ML interests
None yet
Recent Activity
reacted to danielhanchen's post with ❤️ 2 months ago
Kimi K3 can now be run locally! ✨
The 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size).
Run on a Mac Studio connected with 128GB RAM device. Kimi K3 is the strongest open model to date.
GGUF: https://huggingface.co/unsloth/Kimi-K3-GGUF
Guide: https://unsloth.ai/docs/models/kimi-k3 reacted to salma-remyx's post with 🔥 2 months ago
VQASynth adds a code-as-action annotation agent
Data synthesis frameworks may distill a pipeline of expert models through templates or LLM re-phrasing, but templates limit the variety of the training data.
Wanting more flexibility for VQASynth, we've added an experimental SpatialAnnotator agent with scene understanding tools-as-methods, built on NVIDIA-labs OO Agents (NOOA): https://github.com/NVIDIA-NeMo/labs-OO-Agents
In NOOA an agent is a Python class. Fields hold state, docstrings are prompts, type annotations are contracts, and a method body of `...` becomes an LLM-driven loop. Scene understanding tools stay ordinary deterministic methods.
```
from nooa import Agent
class SpatialAnnotator(Agent, llm=llm):
"""Answer spatial questions about an image using scene understanding tools."""
scene: SceneContext # masks, depth, intrinsics, canonical transform
# Deterministic Python. Bodies omitted here.
def measure(self, a, b=None, kind="object_to_object") -> Measurement
def count(self, target: str) -> Count
# Agentic. The runtime hands this to the model.
async def annotate(self, image: Image, question: str) -> SpatialAnswer:
"""Answer the spatial question. This is offline labeling, so there is
no human available for clarification. Prefer a canonical
interpretation, and decline if the question cannot be grounded."""
...
```
GitHub: https://github.com/remyxai/VQASynth#agent-with-vqasynth-toolsOrganizations
None yet
upvoted an article 3 days ago
reacted to danielhanchen's post with ❤️ 2 months ago
Post
5744
Kimi K3 can now be run locally! ✨
The 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size).
Run on a Mac Studio connected with 128GB RAM device. Kimi K3 is the strongest open model to date.
GGUF: unsloth/Kimi-K3-GGUF
Guide: https://unsloth.ai/docs/models/kimi-k3
The 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size).
Run on a Mac Studio connected with 128GB RAM device. Kimi K3 is the strongest open model to date.
GGUF: unsloth/Kimi-K3-GGUF
Guide: https://unsloth.ai/docs/models/kimi-k3
- 5 replies
reacted to salma-remyx's post with 🔥 2 months ago
Post
3079
VQASynth adds a code-as-action annotation agent
Data synthesis frameworks may distill a pipeline of expert models through templates or LLM re-phrasing, but templates limit the variety of the training data.
Wanting more flexibility for VQASynth, we've added an experimental SpatialAnnotator agent with scene understanding tools-as-methods, built on NVIDIA-labs OO Agents (NOOA): https://github.com/NVIDIA-NeMo/labs-OO-Agents
In NOOA an agent is a Python class. Fields hold state, docstrings are prompts, type annotations are contracts, and a method body of
GitHub: https://github.com/remyxai/VQASynth#agent-with-vqasynth-tools
Data synthesis frameworks may distill a pipeline of expert models through templates or LLM re-phrasing, but templates limit the variety of the training data.
Wanting more flexibility for VQASynth, we've added an experimental SpatialAnnotator agent with scene understanding tools-as-methods, built on NVIDIA-labs OO Agents (NOOA): https://github.com/NVIDIA-NeMo/labs-OO-Agents
In NOOA an agent is a Python class. Fields hold state, docstrings are prompts, type annotations are contracts, and a method body of
... becomes an LLM-driven loop. Scene understanding tools stay ordinary deterministic methods.from nooa import Agent
class SpatialAnnotator(Agent, llm=llm):
"""Answer spatial questions about an image using scene understanding tools."""
scene: SceneContext # masks, depth, intrinsics, canonical transform
# Deterministic Python. Bodies omitted here.
def measure(self, a, b=None, kind="object_to_object") -> Measurement
def count(self, target: str) -> Count
# Agentic. The runtime hands this to the model.
async def annotate(self, image: Image, question: str) -> SpatialAnswer:
"""Answer the spatial question. This is offline labeling, so there is
no human available for clarification. Prefer a canonical
interpretation, and decline if the question cannot be grounded."""
...GitHub: https://github.com/remyxai/VQASynth#agent-with-vqasynth-tools
published a
model 2 months ago
updated a
model about 2 years ago