README.md

metadata

title: Script Fidelity Rate
sdk: gradio
app_file: app.py
pinned: false
license: mit
tags:
  - evaluate
  - metric
  - automatic-speech-recognition
  - unicode
  - multilingual-asr

Script Fidelity Rate

This directory is the Hugging Face Evaluate metric module for Script Fidelity Rate (SFR).

The Python package is published as script-fidelity on PyPI: https://pypi.org/project/script-fidelity/. The import name is script_fidelity.

import evaluate

sfr = evaluate.load("./metrics/script_fidelity_rate", module_type="metric")
result = sfr.compute(
    predictions=["کابل کې ښه هوا ده", "this is romanized output"],
    language="ps_af",
)
print(result["sfr_percent"])

Hub use:

import evaluate

sfr = evaluate.load("themechanism/script_fidelity_rate", module_type="metric")
sfr.compute(predictions=["کابل کې ښه هوا ده"], language="pashto")

Use SFR with WER and CER, not instead of them. SFR checks whether output is in the intended script. It does not measure lexical accuracy.