metadata
language: pt
license: mit
tags:
- document-understanding
- document-retrieval
- metric-learning
- siamese-network
- internvl
- cosdoc
datasets:
- LA-CDIP
metrics:
- eer
model-index:
- name: CosDoc (Attention nq=1 (baseline))
results:
- task:
type: document-retrieval
dataset:
name: LA-CDIP
type: la-cdip
metrics:
- type: eer
value: 0.0144
CosDoc — Attention nq=1 (baseline)
CosDoc is a visual document embedding model trained with supervised metric learning and hard-example selection via a Reinforcement Learning professor network.
Pooler variant: Attention nq=1 (baseline) — Standard attention pooler, num_queries=1.
Architecture
| Component | Value |
|---|---|
| Backbone | InternVL3-2B (OpenGVLab/InternVL3-2B) |
| Cut layer | 27 |
| Pooler | attention (num_queries=1) |
| Embedding dim | 1536 |
| Loss | Sub-Center CosFace (m=0.35, s=32, k=3) |
| Embedding prompt | <image> Analyze this document |
Performance (LA-CDIP, full validation pairs)
| Dataset | EER |
|---|---|
| LA-CDIP (5-fold CV) | 1.44% |
Source run: Sprint3b_S0_subcenter_cosface_seed42_s32k3_fase1_E10
Usage
import torch
from huggingface_hub import hf_hub_download
from cavl_doc.models.backbone_loader import load_model
from cavl_doc.models.modeling_cavl import build_cavl_model
device = "cuda" if torch.cuda.is_available() else "cpu"
# Download fine-tuned weights
ckpt_path = hf_hub_download(repo_id="Jpcosta90/cosdoc", filename="best_model.pt")
ckpt = torch.load(ckpt_path, map_location=device, weights_only=False)
cfg = ckpt["config"]
backbone, _, tokenizer, _, _ = load_model("InternVL3-2B")
model = build_cavl_model(
backbone=backbone,
cut_layer=cfg["cut_layer"],
pooler_type=cfg["pooler_type"],
num_queries=cfg.get("num_queries", 1),
)
model.pool.load_state_dict(ckpt["siam_pool"])
model.head.load_state_dict(ckpt["siam_head"])
model.eval().to(device)
Citation
@misc{cosdoc2026,
title = {CosDoc: Cosine-Margin Document Embeddings with RL-guided Hard Mining},
author = {Costa, João Paulo},
year = {2026},
url = {https://huggingface.co/Jpcosta90/cosdoc}
}