Hugging Face's logo

Spaces:
mborcin
/
rca-judgment-validation
Sleeping

App Files Community
rca-judgment-validation / README.md
mborcin's picture
mborcin
Update RCA human evaluation UI
5a2c6b2 verified
preview code
|
Raw
History Blame Contribute Delete
2.52 kB
metadata 
title: RCA Judgment Validation
colorFrom: blue
colorTo: gray
sdk: docker
pinned: false

RCA judgment validation Label Studio Space

This Space runs Label Studio for human cross-validation of machine RCA labels on generated legal judgments.

Required Hugging Face setup

  1. Create the Space as a Docker Space.
  2. Attach a small persistent Storage Bucket mounted at /data.
  3. Add these Space secrets:
    • LABEL_STUDIO_USERNAME: admin email
    • LABEL_STUDIO_PASSWORD: admin password
    • LABEL_STUDIO_USER_TOKEN: stable API token, at least 5 characters
    • SECRET_KEY: stable Django secret key, so sessions survive restarts
  4. Add these Space variables:
    • LABEL_STUDIO_BASE_DATA_DIR=/data
    • STORAGE_PERSISTENCE=1
    • LABEL_STUDIO_ENABLE_LEGACY_API_TOKEN=true
    • LABEL_STUDIO_DISABLE_SIGNUP_WITHOUT_LINK=true

The bootstrap creates three projects with ten tasks each:

  • RCA Validation - Annotator 1
  • RCA Validation - Annotator 2
  • RCA Validation - Annotator 3

Each task shows the case metadata, generated/reference outcomes, model input case record, machine RCA, and generated/reference judgments side-by-side. Annotators answer:

  • Rule stage
  • Major error
  • Minor error

Local data generation

From the repository root:

python3 label_studio_space/scripts/prepare_tasks.py

This regenerates:

  • label_studio_space/data/tasks_all.json
  • label_studio_space/data/tasks_annotator_1.json
  • label_studio_space/data/tasks_annotator_2.json
  • label_studio_space/data/tasks_annotator_3.json

Export annotations

After annotation, use Label Studio's project export UI, or call:

curl -H "Authorization: Token $LABEL_STUDIO_USER_TOKEN" \
  "$SPACE_URL/api/projects/<project_id>/export?exportType=JSON"

Programmatic HF provisioning

If huggingface_hub is installed and authenticated, provision the bucket mount, variables, secrets, and optional factory rebuild with:

python3 label_studio_space/scripts/provision_hf_space.py \
  --space-id <namespace>/<space> \
  --bucket-id <namespace>/label-studio-data \
  --admin-email <admin@example.com> \
  --admin-password <password> \
  --admin-token <stable-token> \
  --factory-reboot

To create/upload/provision/restart in one step:

python3 label_studio_space/scripts/deploy_hf_space.py \
  --space-id <namespace>/<space> \
  --bucket-id <namespace>/label-studio-data \
  --admin-email <admin@example.com> \
  --admin-password <password> \
  --admin-token <stable-token>