|
|
--- |
|
|
language: en |
|
|
tags: |
|
|
- multiple-instance-learning |
|
|
- fraud-detection |
|
|
- risk-assessment |
|
|
- anomaly-detection |
|
|
- graph-transformer |
|
|
- interpretability |
|
|
license: apache-2.0 |
|
|
library_name: pytorch |
|
|
--- |
|
|
|
|
|
# AC-MIL |
|
|
|
|
|
AC-MIL (Action-Aware Capsule Multiple Instance Learning for Live-Streaming Room Risk Assessment) is a weakly supervised model for **room-level risk assessment** in live-streaming platforms. It is designed for scenarios where only **binary room-level labels** are available, while risk evidence is often **sparse, localized, and manifested through coordinated behaviors** across users and time. |
|
|
|
|
|
AC-MIL formulates each live room as a **Multiple Instance Learning (MIL)** bag, where each instance is a **user–timeslot capsule**—a short action subsequence performed by a particular user within a fixed time window. The model produces: |
|
|
- a **room-level risk score**, indicating the probability that the room is risky, and |
|
|
- **capsule-level attributions**, providing interpretable evidence by highlighting suspicious user–time segments that contribute most to the prediction. |
|
|
|
|
|
|
|
|
--- |
|
|
|
|
|
## Key idea |
|
|
|
|
|
Given a room’s action stream, we construct a 2D grid of capsules over **users × timeslots**. Each capsule summarizes localized behavioral patterns within a specific user–time window. AC-MIL then models: |
|
|
- **temporal dynamics**: how users’ behaviors evolve over time, |
|
|
- **cross-user dependencies**: interactions between viewers and the streamer, as well as coordination patterns among viewers, |
|
|
- **multi-level signals**: evidence captured at the action, capsule, user, and timeslot levels, |
|
|
and fuses these signals to produce robust room-level risk predictions. |
|
|
|
|
|
--- |
|
|
|
|
|
## Architecture overview |
|
|
|
|
|
AC-MIL follows a hierarchical serial–parallel design: |
|
|
|
|
|
<p align="center"> |
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/671b2ede3fd1d03dc687c641/Vry7GWMLjGKLsCBqq5B6g.png" width="80%"/> |
|
|
<p> |
|
|
|
|
|
|
|
|
1. **Action Field Encoder** |
|
|
- Encodes the full action sequence with a Transformer to produce contextualized action embeddings. |
|
|
- Produces an action-level room representation via a learnable `[CLS]` token. |
|
|
|
|
|
2. **Capsule Constructor** |
|
|
- Partitions actions into **user–timeslot capsules**. |
|
|
- Encodes each capsule with an LSTM (final hidden state as capsule embedding). |
|
|
|
|
|
3. **Relational Capsule Reasoner** |
|
|
- Builds an **adaptive relation-aware graph** over capsules using emantic similarity and relation masks. |
|
|
- Runs a **graph-aware Transformer** to refine capsule embeddings. |
|
|
- Provides **capsule-level interpretability** via `[CLS] → capsule` attention. |
|
|
|
|
|
4. **Dual-View Integrator** |
|
|
- **User-view**: GRU over each user’s capsule sequence and attention pooling across users. |
|
|
- **Timeslot-view**: attention pooling within each timeslot and GRU across timeslots. |
|
|
|
|
|
5. **Cross-Level Risk Decoder** |
|
|
- Learns gates over multi-level room representations. |
|
|
- Produces the final room embedding and risk score. |
|
|
|
|
|
|
|
|
--- |
|
|
|
|
|
|
|
|
## Input / output specification |
|
|
|
|
|
### Input (conceptual) |
|
|
|
|
|
Each dataset sample corresponds to a live room `room_id` with a binary room-level label `room_label ∈ {0,1,2,3}` (>0, risky). |
|
|
A room is represented as an **action sequence** `patch_list = {α_i}` ordered by tuple (user, time), where each action follows the paper’s definition: |
|
|
`α = (u, t, a, x)` (user, timestamp, action type, and optional textual/multimodal feature). |
|
|
|
|
|
In our May/June datasets, each action record is stored with the following fields: |
|
|
|
|
|
- `u_idx` (int): user index within the room, used to build the `users × timeslots` grid (e.g., 0 = streamer, 1..U = selected viewers). |
|
|
- `global_user_idx` (int/str): global user identifier across the whole dataset (before remapping to `u_idx`). |
|
|
- `timestamp` (int/float): the action timestamp`t`. In the formulation, timestamps are within a window `[0, T]` after the room starts. |
|
|
- `t` (int): timeslot index derived by discretizing `timestamp` into fixed-length windows. This is the column index when constructing the `users × timeslots` capsule grid. |
|
|
- `l` (int): role indicator (recommended convention: `0 = viewer`, `1 = streamer`). |
|
|
- `action_id` (int): the action type id `a` (e.g., enter, comment, like, gift, share; streamer-side actions may include stream start, ASR text, OCR text, etc.). |
|
|
- `action_desc` (str / null): raw textual content associated with the action (e.g., comment text, ASR transcript, OCR text). |
|
|
- `action_vec` (numpy): pre-encoded feature vector for `action_desc`. |
|
|
|
|
|
|
|
|
Example (JSONL-like): |
|
|
```json |
|
|
{ |
|
|
"room_id": "1", |
|
|
"room_label": "2", |
|
|
"patch_list": [ |
|
|
(u_idx, t, l, action_id, action_vec, timestamp, action_desc, user_id), |
|
|
(0, 1, 0, 5, [0.0, 0.3, ...], 4, "主播口播:...", 5415431), |
|
|
... |
|
|
] |
|
|
} |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Intended use |
|
|
|
|
|
**Primary use cases** |
|
|
- Early detection of risky rooms (fraud, collusion, policy-violating coordinated behaviors) |
|
|
- Evidence-based moderation: highlight localized suspicious segments (user–time capsules) |
|
|
|
|
|
**Out of scope** |
|
|
- Identifying or tracking specific individuals |
|
|
- Any use that violates privacy laws, platform policies, or user consent requirements |
|
|
|
|
|
--- |
