abhshkp/footfall-analysis-vit-stage1

ViT Gender Classifier (Stage 1)

This repository contains the stage-1 weights for a fine-tuned Vision Transformer (ViT) model optimized for binary gender classification (female and male). The model is designed to be integrated into footfall analysis, retail analytics, and demographic tracking pipelines.

Model Description

The model utilizes the google/vit-base-patch16-224 architecture as its backbone, which was pre-trained on ImageNet-21k at a resolution of 224x224 pixels. It wraps the ViT backbone in a custom PyTorch class (GenderClassifier) that extracts the representation of the [CLS] token and passes it to a linear classifier layer to produce binary logits.

• Developer: [Your Name / Organization]
• Model Type: Image Classification (Vision Transformer backbone)
• Base Model: google/vit-base-patch16-224
• Language(s): English (API / Documentation)
• License: Apache 2.0

Intended Uses & Limitations

Intended Use Cases

1. Footfall Analysis & Retail Analytics: Identifying gender distribution trends over time in public spaces, retail stores, or venues.
2. Audience Demographics: Anonymized demographic statistics for digital signage, event registration spaces, or public plazas.
3. Pipeline Integration: Designed to follow an upstream person-detector (e.g., YOLO, Faster R-CNN) and person-tracker (e.g., DeepSORT, ByteTrack), classifying cropped bounding boxes of detected individuals.

Limitations

• Binary Assumption: The model restricts classification to binary categories (female and male), which does not capture the full diversity of gender identities.
• Image Quality Dependencies: Accuracy decreases significantly with low-resolution crops, motion blur, extreme angles (e.g., top-down security cameras), or heavy occlusions.
• Demographic Bias: Vision-based gender classification models can exhibit performance disparities across different ages, ethnicities, and lighting conditions.

How to Get Started (Inference)

Below is a detailed guide on how to preprocess inputs and run inference using PyTorch and the Hugging Face transformers library.

1. Model Architecture Definition

Ensure the custom GenderClassifier class matches the structure expected by the saved pytorch_model.bin weights:

import torch
import torch.nn as nn
from transformers import ViTModel

class GenderClassifier(nn.Module):
    def __init__(self, config):
        super().__init__()
        self.vit = ViTModel.from_pretrained(config.vit_model_name)
        self.classifier = nn.Linear(self.vit.config.hidden_size, config.num_labels)
        
    def forward(self, pixel_values):
        outputs = self.vit(pixel_values=pixel_values)
        # Extract the sequence output for the CLS token
        sequence_output = outputs.last_hidden_state[:, 0, :]
        # Pass to classification head
        logits = self.classifier(sequence_output)
        return logits

2. Loading the Model

import torch
from transformers import ViTConfig

# Path to the directory containing config.json and pytorch_model.bin
model_path = "./path_to_model_directory"

# 1. Load the configuration
config = ViTConfig.from_pretrained(model_path)

# 2. Instantiate the model skeleton
model = GenderClassifier(config)

# 3. Load the fine-tuned weights
model.load_state_dict(torch.load(f"{model_path}/pytorch_model.bin", map_location="cpu"))
model.eval()

3. Preprocessing and Inference Pipeline

from PIL import Image
import torch
import torch.nn.functional as F
from transformers import ViTImageProcessor

# Initialize the image processor corresponding to the base ViT model
processor = ViTImageProcessor.from_pretrained("google/vit-base-patch16-224")

# Load and preprocess an image (crop of a person)
image_path = "path_to_person_crop.jpg"
image = Image.open(image_path).convert("RGB")

# Preprocess image to tensor matching expected resolution (224x224) and normalization
inputs = processor(images=image, return_tensors="pt")

# Run inference
with torch.no_grad():
    logits = model(pixel_values=inputs.pixel_values)
    probabilities = F.softmax(logits, dim=-1)
    predicted_class_id = torch.argmax(probabilities, dim=-1).item()

# Map prediction to label
id2label = config.id2label  # {0: "female", 1: "male"}
print(f"Predicted class: {id2label[str(predicted_class_id)]} ({probabilities[0][predicted_class_id].item():.4f})")

Training and Evaluation Details (Suggested Setup)

Training Data

The model was fine-tuned on person crops extracted from pedestrian detection datasets, annotated with binary gender labels.

Training Procedure

• Optimizer: AdamW
• Learning Rate: 2e-5 with linear warmup and decay
• Loss Function: Cross-Entropy Loss
• Batch Size: 32 or 64
• Precision: Mixed Precision (FP16)

Evaluation Metrics

• Accuracy: The proportion of correctly classified instances.
• Precision / Recall / F1-Score: Monitored per class to ensure balanced predictions and minimal bias.

Ethical Considerations

When deploying automated gender classification models, care must be taken to:

• Prioritize Privacy: Do not store original face or body images. Process images in-memory and store only aggregated count statistics (e.g., "55% female, 45% male").
• Transparency: Disclose the use of automated demographic tracking to individuals in the monitored environment where legally required.