README.md

---
license: apache-2.0
language:
  - ase
pipeline_tag: video-classification
tags:
  - asl
  - sign-language
  - accessibility
  - autonomous-vehicles
  - computer-vision
  - human-intent
  - multimodal
  - unmute1ai
  - signal-av
metrics:
  - accuracy
  - f1
---

# SIGNAL-AV

**Accessible human-intent perception for autonomous vehicles.**

SIGNAL-AV is an Unmute1AI project designed to translate visual human communication — beginning with American Sign Language (ASL) and safety-critical gestures — into structured, machine-readable intent for autonomous vehicle systems.

## Mission

Autonomous vehicles can detect people.

SIGNAL-AV is designed to help them understand when people are intentionally communicating with them.

**Making Every Signal Accessible to All.**

## Core Flow

```text
Camera / Visual Input
        ↓
Hand + Pose + Facial Features
        ↓
Temporal Sign Analysis
        ↓
Intent Recognition
        ↓
Confidence + Spatial Context
        ↓
Safety Policy Gate
        ↓
Structured Vehicle Intent
```

## Initial Intent Vocabulary

- STOP
- GO
- WAIT
- HELP
- DANGER
- COME
- LEFT
- RIGHT
- UNKNOWN

## Example Output

```json
{
  "intent": "STOP",
  "source": "pedestrian_sign",
  "language": "ASL",
  "confidence": 0.94,
  "spatial_relation": "front_right",
  "safety_class": "advisory"
}
```

## Safety Architecture

SIGNAL-AV does not directly control vehicle motion.

It produces a structured human-intent observation that can be evaluated alongside vehicle perception, localization, trajectory planning, environmental context, and safety policies.

Ambiguous or low-confidence communication should resolve to `UNKNOWN` rather than forcing an interpretation.

## Design Principles

### Accessibility First

Human communication should not become invisible simply because it is visual instead of spoken.

### Intent Before Language

SIGNAL-AV is designed to identify meaning and intent rather than perform simple word-for-word translation.

### Multimodal Understanding

SIGNAL-AV is designed to reason across:

- hand movement
- hand shape
- body pose
- facial expression
- temporal motion
- spatial context

### Safety-Constrained Output

Recognized communication becomes an advisory perception signal rather than an unrestricted vehicle command.

### Local-First Architecture

The architecture is designed for edge processing where practical to reduce latency and unnecessary transmission of raw visual data.

## Intended Use Cases

- Deaf and Hard-of-Hearing pedestrian communication
- ASL interaction with autonomous vehicles
- traffic-control gestures
- emergency communication
- accessibility-aware robotaxis
- autonomous shuttles
- smart transportation systems
- human-to-robot communication

## Architecture Goal

```text
HUMAN
  ↓
Visual Signal
  ↓
SIGNAL-AV
  ↓
Structured Intent
  ↓
Safety + Confidence Validation
  ↓
AV Reasoning System
  ↓
Vehicle Response
```

## Status

**Early development / research prototype.**

Current priorities:

1. Define the intent schema
2. Build the temporal visual perception pipeline
3. Establish ASL and gesture evaluation datasets
4. Implement confidence and UNKNOWN handling
5. Create simulation adapters
6. Benchmark recognition latency and accuracy
7. Validate accessibility and safety behavior

## Unmute1AI

SIGNAL-AV is part of the Unmute1AI accessibility ecosystem.

**Accessibility First. Always.**

**Making Every Signal Accessible to All.**