README.md

---
license: other
license_name: edgeface-idiap
license_link: https://gitlab.idiap.ch/bob/bob.paper.tbiom2023_edgeface/-/blob/master/LICENSE
library_name: transformers
pipeline_tag: image-feature-extraction
tags:
  - face-recognition
  - face-verification
  - face-embedding
  - edgeface
  - timm
---

# EdgeFace for 🤗 Transformers

[EdgeFace](https://arxiv.org/abs/2307.01838) (Idiap Research Institute) packaged as a `transformers` custom model. All four published variants live in this single repository as subfolders and are accessible through the standard `AutoModel` / `AutoImageProcessor` API with built-in MediaPipe face alignment.

EdgeFace replaces the classifier of an `edgenext` (timm) backbone with a 512-d embedding head trained for face recognition. Two variants additionally apply a static low-rank factorization to their linear layers — EdgeFace's "gamma" trick, baked into the pretrained weights and unrelated to PEFT adapters.

## Model variants

| Subfolder | Backbone | Low-rank ratio | Params | |
|---|---|---|---|---|
| `edgeface-base` | `edgenext_base` | — | ~18 M | **default** |
| `edgeface-s-gamma-05` | `edgenext_small` | 0.5 | ~5 M | |
| `edgeface-xs-gamma-06` | `edgenext_x_small` | 0.6 | ~3 M | |
| `edgeface-xxs` | `edgenext_xx_small` | — | ~1 M | |

## Installation

```bash
pip install transformers timm torch safetensors huggingface_hub numpy

# Face alignment (do_align=True) also requires:
pip install mediapipe opencv-python
```

## Quick start

### Pipeline (default — edgeface-xxs)

```python
from transformers import pipeline

pipe = pipeline("image-feature-extraction", model="anjith2006/edgeface", trust_remote_code=True)
```

### AutoModel (any variant)

```python
import torch
import torch.nn.functional as F
from PIL import Image
from transformers import AutoModel, AutoImageProcessor

repo = "anjith2006/edgeface"
variant = "edgeface-xxs"   # or edgeface-base / edgeface-s-gamma-05 / edgeface-xs-gamma-06

model = AutoModel.from_pretrained(repo, subfolder=variant, trust_remote_code=True).eval()
processor = AutoImageProcessor.from_pretrained(repo, subfolder=variant, trust_remote_code=True)

@torch.no_grad()
def embed(path):
    img = Image.open(path).convert("RGB")
    inputs = processor(img, return_tensors="pt")   # do_align=True by default
    return F.normalize(model(**inputs).embeddings, dim=-1)

score = F.cosine_similarity(embed("a.jpg"), embed("b.jpg")).item()
print(f"{score:.4f}")  # → ~0.9+ same person, lower for different
```

## Face alignment

The image processor detects and aligns the face by default, warping it onto the ArcFace 112×112 template using 5 MediaPipe landmarks — the same alignment the weights were trained with.

```python
# Full image → detect face, align, normalize (default)
inputs = processor(img, return_tensors="pt")

# Pre-aligned 112×112 crop → skip detection, just normalize
inputs = processor(crop, do_align=False, return_tensors="pt")

# Known landmarks → skip detection, align from provided 5 points
inputs = processor(img, landmarks=pts, return_tensors="pt")  # pts: ndarray (5, 2)
```

If detection fails the processor falls back to a plain resize so batches never crash.

### MediaPipe backend

```python
# "auto" (default): try the Tasks API, fall back to legacy solutions.face_mesh
# "tasks":          force the modern API — downloads face_landmarker.task once to ~/.cache/edgeface/
# "solutions":      force the legacy API (older mediapipe installs)
processor = AutoImageProcessor.from_pretrained(
    repo, subfolder=variant, trust_remote_code=True, mp_backend="tasks"
)

# Offline / custom bundle:
processor = AutoImageProcessor.from_pretrained(
    repo, subfolder=variant, trust_remote_code=True,
    mp_model_path="/path/to/face_landmarker.task"
)
# or: export EDGEFACE_MP_MODEL=/path/to/face_landmarker.task
```

## Batch usage

```python
imgs = [Image.open(p).convert("RGB") for p in paths]
inputs = processor(imgs, return_tensors="pt")
with torch.no_grad():
    embs = F.normalize(model(**inputs).embeddings, dim=-1)  # (N, 512)
```

## Local import without `trust_remote_code`

Clone the source repo and import the package directly:

```python
from edgeface import register_edgeface
register_edgeface()  # wires EdgeFace into AutoConfig / AutoModel / AutoImageProcessor

model = AutoModel.from_pretrained("anjith2006/edgeface", subfolder="edgeface-xxs").eval()
processor = AutoImageProcessor.from_pretrained("anjith2006/edgeface", subfolder="edgeface-xxs")
```

## LoRA fine-tuning

The static low-rank layers in the gamma variants are plain `nn.Linear` modules, so PEFT targets them without any naming collision:

```python
from peft import LoraConfig, get_peft_model

model = AutoModel.from_pretrained(repo, subfolder=variant, trust_remote_code=True)

# Gamma variants (edgeface-s-gamma-05, edgeface-xs-gamma-06):
lora_cfg = LoraConfig(r=8, lora_alpha=16, target_modules=["linear1", "linear2"])

# Base / XXS variants (no factorized layers — target the backbone linears directly):
# print([n for n, _ in model.named_modules() if isinstance(_, torch.nn.Linear)])
lora_cfg = LoraConfig(r=8, lora_alpha=16, target_modules=["fc1", "fc2"])

model = get_peft_model(model, lora_cfg)
model.print_trainable_parameters()
```

## Source files

| File | Purpose |
|---|---|
| `configuration_edgeface.py` | `EdgeFaceConfig` |
| `modeling_edgeface.py` | `EdgeFaceModel`, `LowRankLinear`, `EdgeFaceOutput` |
| `image_processing_edgeface.py` | `EdgeFaceImageProcessor` (MediaPipe alignment + normalize) |
| `convert_edgeface.py` | Download original `.pt` checkpoints, convert, push |
| `example.py` | Same-person / different-person sanity check |

## License

The pretrained weights and original alignment code are © Idiap Research Institute. The original [EdgeFace license](https://gitlab.idiap.ch/bob/bob.paper.tbiom2023_edgeface/-/blob/master/LICENSE) governs all weight files and derivative uses. See `NOTICE` for details. Verify compliance before commercial use or redistribution.

## Citation

```bibtex
@article{george2024edgeface,
  title   = {EdgeFace: Efficient Face Recognition Model for Edge Devices},
  author  = {George, Anjith and Ecabert, Christophe and Otroshi Shahreza, Hatef
             and Kotwal, Ketan and Marcel, Sebastien},
  journal = {IEEE Transactions on Biometrics, Behavior, and Identity Science},
  year    = {2024},
  doi     = {10.1109/TBIOM.2024.3352169}
}
```