Improve model card: add pipeline tag, links, and clean metadata
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -3,9 +3,10 @@ license: other
|
|
| 3 |
license_name: license
|
| 4 |
license_link: LICENSE
|
| 5 |
spaces:
|
| 6 |
-
|
| 7 |
-
|
| 8 |
---
|
|
|
|
| 9 |
<a name="readme-top"></a>
|
| 10 |
|
| 11 |
<div align="center" style="background-color: #0e2841; padding: 20px; border-radius: 15px; margin-bottom: 20px;">
|
|
@@ -15,6 +16,7 @@ arxiv: 2605.00271
|
|
| 15 |
|
| 16 |
Welcome to **REALM**! This repository contains the implementation of REALM, for advanced computer vision tasks involving both traditional RGB and Event-based vision.
|
| 17 |
|
|
|
|
| 18 |
|
| 19 |
<div align="center">
|
| 20 |
<img src="media/demo_realm.gif" alt="demo" >
|
|
@@ -68,7 +70,7 @@ Event cameras provide several unique advantages over standard frame-based sensor
|
|
| 68 |
|
| 69 |
We address this gap with **REALM**, a cross-modal framework that learns an **R**GB and **E**vent **A**ligned **L**atent **M**anifold by projecting event representations into the pretrained latent space of RGB foundation models. Instead of task-specific training, we leverage low-rank adaptation (LoRA) to bridge the modality gap, effectively unlocking the geometric and semantic priors of frozen RGB backbones for asynchronous event streams.
|
| 70 |
|
| 71 |
-
We demonstrate that **REALM** effectively maps events into the ViT-based foundation latent space. Our method allows us to perform downstream tasks like depth estimation and semantic segmentation by simply transferring linear heads trained on the RGB teacher. Most significantly, **REALM** enables the direct, zero-shot application of complex, frozen image-trained decoders, such as MASt3R, to raw event data. We demonstrate state-of-the-art performance in wide-baseline feature matching, significantly outperforming specialized architectures.
|
| 72 |
|
| 73 |
<p align="right">(<a href="#readme-top">back to top</a>)</p>
|
| 74 |
|
|
|
|
| 3 |
license_name: license
|
| 4 |
license_link: LICENSE
|
| 5 |
spaces:
|
| 6 |
+
- viciopoli/REALM-demo
|
| 7 |
+
pipeline_tag: image-feature-extraction
|
| 8 |
---
|
| 9 |
+
|
| 10 |
<a name="readme-top"></a>
|
| 11 |
|
| 12 |
<div align="center" style="background-color: #0e2841; padding: 20px; border-radius: 15px; margin-bottom: 20px;">
|
|
|
|
| 16 |
|
| 17 |
Welcome to **REALM**! This repository contains the implementation of REALM, for advanced computer vision tasks involving both traditional RGB and Event-based vision.
|
| 18 |
|
| 19 |
+
[๐ Paper](https://huggingface.co/papers/2605.00271) | [๐ป GitHub](https://github.com/utiasSTARS/REALM) | [๐ Project Page](https://papers.starslab.ca/realm/) | [๐ค Demo](https://huggingface.co/spaces/viciopoli/REALM-demo)
|
| 20 |
|
| 21 |
<div align="center">
|
| 22 |
<img src="media/demo_realm.gif" alt="demo" >
|
|
|
|
| 70 |
|
| 71 |
We address this gap with **REALM**, a cross-modal framework that learns an **R**GB and **E**vent **A**ligned **L**atent **M**anifold by projecting event representations into the pretrained latent space of RGB foundation models. Instead of task-specific training, we leverage low-rank adaptation (LoRA) to bridge the modality gap, effectively unlocking the geometric and semantic priors of frozen RGB backbones for asynchronous event streams.
|
| 72 |
|
| 73 |
+
We demonstrate that **REALM** effectively maps events into the ViT-based foundation latent space. Our method allows us to perform downstream tasks like depth estimation and semantic segmentation by simply transferring linear heads trained on the RGB teacher. Most significantly, **REALM** enables the direct, zero-shot application of complex, frozen image-trained decoders, such as MASt3R, to raw event data. We demonstrate state-of-the-art performance in wide-baseline feature matching, significantly outperforming specialized architectures.
|
| 74 |
|
| 75 |
<p align="right">(<a href="#readme-top">back to top</a>)</p>
|
| 76 |
|