From Fragmentation to Fusion: A Survey and Roadmap for Omnimodal Embeddings

Despite shared mathematical foundations, embedding research across text, vision, audio, graphs, tables, and time series remains largely fragmented into modality-specific silos. This survey and position paper argues that embedding research is quietly converging around universal principles: contrastive learning objectives, Transformer backbones, the pretrain–finetune recipe, and an upward shift toward whole semantic units. To evaluate progress toward true unification, we introduce the 3C framework—Coverage, Comparability, and Compositionality—which maps onto three progressive paths: format transformation, cross-modal alignment, and deep joint fusion, our advocated terminus that subsumes alignment and eliminates the modality gap. Reaching this goal will require preserving unique modality-specific structures, cross-pollinating techniques across research communities, and building unified benchmarks that measure true compositional gains.