ASR Benchmark Report

Date: 2026-03-09 11:06:14
Experiments: 22
Engines: 11

Metrics Heatmap

Comparison of all metrics across engine/condition combinations. Green = better, Red = worse. WIP is inverted (higher is better).

Resilience Analysis (Box Plots)

Analyze performance distribution and resilience across different groupings.

parakeet-rnnt-1.1b • None • Raw
WER: 22.51% CER: 5.89% MER: 22.22% WIL: 38.72% WIP: 61.28% Time: 13.21s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-rnnt-1.1b • None • Normalized
WER: 2.60% CER: 0.24% MER: 2.56% WIL: 3.83% WIP: 96.17% Time: 13.21s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-ctc-1.1b • None • Raw
WER: 22.51% CER: 5.89% MER: 22.22% WIL: 38.72% WIP: 61.28% Time: 23.90s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-ctc-1.1b • None • Normalized
WER: 2.60% CER: 0.24% MER: 2.56% WIL: 3.83% WIP: 96.17% Time: 23.90s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-rnnt-0.6b • None • Raw
WER: 22.51% CER: 5.89% MER: 22.22% WIL: 38.72% WIP: 61.28% Time: 17.14s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-rnnt-0.6b • None • Normalized
WER: 2.60% CER: 0.24% MER: 2.56% WIL: 3.83% WIP: 96.17% Time: 17.14s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-ctc-0.6b • None • Raw
WER: 22.94% CER: 6.28% MER: 22.65% WIL: 39.39% WIP: 60.61% Time: 5.58s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hight week the flat was seven flights up and winston who was thirty nine and had a varicoses ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-ctc-0.6b • None • Normalized
WER: 3.46% CER: 0.64% MER: 3.42% WIL: 5.51% WIP: 94.49% Time: 5.58s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hight week the flat was seven flights up and winston who was thirty nine and had a varicoses ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-tdt-1.1b • None • Raw
WER: 22.51% CER: 5.89% MER: 22.22% WIL: 38.72% WIP: 61.28% Time: 17.50s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-tdt-1.1b • None • Normalized
WER: 2.60% CER: 0.24% MER: 2.56% WIL: 3.83% WIP: 96.17% Time: 17.50s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-tdt_ctc-1.1b • None • Raw
WER: 6.93% CER: 2.59% MER: 6.84% WIL: 12.08% WIP: 87.92% Time: 9.74s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
It was a bright cold day in April and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a meter wide the face of a man of about forty five, with a heavy black mustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty nine and had a Varico's ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. Big brother is watching you, the caption beneath it ran.
parakeet-tdt_ctc-1.1b • None • Normalized
WER: 3.90% CER: 0.56% MER: 3.85% WIL: 6.34% WIP: 93.66% Time: 9.74s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a meter wide the face of a man of about forty five with a heavy black mustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicos ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-tdt_ctc-110m • None • Raw
WER: 7.36% CER: 2.43% MER: 7.26% WIL: 12.89% WIP: 87.11% Time: 2.72s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of victory mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide, the face of a man of about forty five with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for hate week. The flat was seven flights up, and Winston, who was thirty nine and had a Varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. Big Brother is watching you, the caption beneath it ran.
parakeet-tdt_ctc-110m • None • Normalized
WER: 2.60% CER: 0.24% MER: 2.56% WIL: 3.83% WIP: 96.17% Time: 2.72s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-tdt-0.6b-v2 • None • Raw
WER: 5.63% CER: 2.43% MER: 5.60% WIL: 10.51% WIP: 89.49% Time: 5.94s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it, a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide, the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times, it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for hate week. The flat was seven flights up, and Winston, who was thirty-nine and had a Varico's ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. Big brother is watching you, the caption beneath it ran.
parakeet-tdt-0.6b-v2 • None • Normalized
WER: 1.30% CER: 0.16% MER: 1.29% WIL: 2.15% WIP: 97.85% Time: 5.94s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicos ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
parakeet-tdt-0.6b-v3 • None • Raw
WER: 6.49% CER: 2.28% MER: 6.41% WIL: 11.27% WIP: 88.73% Time: 6.55s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a meter wide, the face of a man of about forty five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for hate week. The flat was seven flights up, and Winston, who was thirty nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. Big Brother is watching you, the caption beneath it ran.
parakeet-tdt-0.6b-v3 • None • Normalized
WER: 3.03% CER: 0.40% MER: 2.99% WIL: 4.67% WIP: 95.33% Time: 6.55s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a meter wide the face of a man of about forty five with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the lift shaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
canary-1b • None • Raw
WER: 57.14% CER: 54.79% MER: 56.90% WIL: 60.65% WIP: 39.35% Time: 221.17s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The flat was seven flights up, and Winston, who was thirty nine and had a varicose ulcer above his right ankle, was tacked to the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. Big Brother is watching you, the caption beneath it ran.
canary-1b • None • Normalized
WER: 54.98% CER: 52.94% MER: 54.74% WIL: 56.61% WIP: 43.39% Time: 221.17s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the flat was seven flights up and winston who was thirty nine and had a varicose ulcer above his right ankle was tacked to the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
canary-1b-flash • None • Raw
WER: 53.25% CER: 53.45% MER: 53.25% WIL: 55.32% WIP: 44.68% Time: 64.69s
Reference
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.
Hypothesis (Diff)
It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. Big Brother is watching you, the caption beneath it ran.
canary-1b-flash • None • Normalized
WER: 51.08% CER: 51.57% MER: 51.08% WIL: 51.08% WIP: 48.92% Time: 64.69s
Reference
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it depicted simply an enormous face more than a metre wide the face of a man of about fortyfive with a heavy black moustache and ruggedly handsome features winston made for the stairs it was no use trying the lift even at the best of times it was seldom working and at present the electric current was cut off during daylight hours it was part of the economy drive in preparation for hate week the flat was seven flights up and winston who was thirtynine and had a varicose ulcer above his right ankle went slowly resting several times on the way on each landing opposite the liftshaft the poster with the enormous face gazed from the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Hypothesis (Diff)
it was a bright cold day in april and the clocks were striking thirteen winston smith his chin nuzzled into his breast in an effort to escape the vile wind slipped quickly through the glass doors of victory mansions though not quickly enough to prevent a swirl of gritty dust from entering along with him the hallway smelt of boiled cabbage and old rag mats at one end of it a coloured poster too large for indoor display had been tacked to the wall it was one of those pictures which are so contrived that the eyes follow you about when you move big brother is watching you the caption beneath it ran
Engine Degradation Enhancement Audio Norm Text Norm WER CER MER WIL WIP Time (s)
parakeet-rnnt-1.1b None - - Raw 0.2251 0.0589 0.2222 0.3872 0.6128 13.21
parakeet-rnnt-1.1b None - - Normalized 0.0260 0.0024 0.0256 0.0383 0.9617 13.21
parakeet-ctc-1.1b None - - Raw 0.2251 0.0589 0.2222 0.3872 0.6128 23.90
parakeet-ctc-1.1b None - - Normalized 0.0260 0.0024 0.0256 0.0383 0.9617 23.90
parakeet-rnnt-0.6b None - - Raw 0.2251 0.0589 0.2222 0.3872 0.6128 17.14
parakeet-rnnt-0.6b None - - Normalized 0.0260 0.0024 0.0256 0.0383 0.9617 17.14
parakeet-ctc-0.6b None - - Raw 0.2294 0.0628 0.2265 0.3939 0.6061 5.58
parakeet-ctc-0.6b None - - Normalized 0.0346 0.0064 0.0342 0.0551 0.9449 5.58
parakeet-tdt-1.1b None - - Raw 0.2251 0.0589 0.2222 0.3872 0.6128 17.50
parakeet-tdt-1.1b None - - Normalized 0.0260 0.0024 0.0256 0.0383 0.9617 17.50
parakeet-tdt_ctc-1.1b None - - Raw 0.0693 0.0259 0.0684 0.1208 0.8792 9.74
parakeet-tdt_ctc-1.1b None - - Normalized 0.0390 0.0056 0.0385 0.0634 0.9366 9.74
parakeet-tdt_ctc-110m None - - Raw 0.0736 0.0243 0.0726 0.1289 0.8711 2.72
parakeet-tdt_ctc-110m None - - Normalized 0.0260 0.0024 0.0256 0.0383 0.9617 2.72
parakeet-tdt-0.6b-v2 None - - Raw 0.0563 0.0243 0.0560 0.1051 0.8949 5.94
parakeet-tdt-0.6b-v2 None - - Normalized 0.0130 0.0016 0.0129 0.0215 0.9785 5.94
parakeet-tdt-0.6b-v3 None - - Raw 0.0649 0.0228 0.0641 0.1127 0.8873 6.55
parakeet-tdt-0.6b-v3 None - - Normalized 0.0303 0.0040 0.0299 0.0467 0.9533 6.55
canary-1b None - - Raw 0.5714 0.5479 0.5690 0.6065 0.3935 221.17
canary-1b None - - Normalized 0.5498 0.5294 0.5474 0.5661 0.4339 221.17
canary-1b-flash None - - Raw 0.5325 0.5345 0.5325 0.5532 0.4468 64.69
canary-1b-flash None - - Normalized 0.5108 0.5157 0.5108 0.5108 0.4892 64.69

Metrics Reference

This section explains the metrics used to evaluate ASR (Automatic Speech Recognition) performance. All metrics are computed by comparing the reference (ground truth) text with the hypothesis (transcription) text.

WER - Word Error Rate

Range: 0 to ∞ (typically 0 to 1)  |  Lower is better

The most common ASR metric. Measures the edit distance at word level.

WER = (Substitutions + Deletions + Insertions) / Total Reference Words

Example: If reference is "the cat sat" and hypothesis is "a cat sits", WER = 2/3 = 0.667

CER - Character Error Rate

Range: 0 to ∞ (typically 0 to 1)  |  Lower is better

Like WER but computed at character level. More granular, useful for languages without clear word boundaries.

CER = (Char Substitutions + Char Deletions + Char Insertions) / Total Reference Characters

Note: CER is often lower than WER since partial word matches are credited.

MER - Match Error Rate

Range: 0 to 1  |  Lower is better

Proportion of words that were not correctly matched between reference and hypothesis.

MER = (Substitutions + Deletions + Insertions) / (Hits + Substitutions + Deletions + Insertions)

Key difference from WER: MER is bounded at 1.0 and accounts for hypothesis length.

WIL - Word Information Lost

Range: 0 to 1  |  Lower is better

Measures the proportion of word information that was lost in transcription.

WIL = 1 - (Hits² / (Reference Length × Hypothesis Length))

Interpretation: Combines precision and recall into a single information-theoretic measure.

WIP - Word Information Preserved

Range: 0 to 1  |  Higher is better

The complement of WIL - measures how much word information was correctly preserved.

WIP = Hits² / (Reference Length × Hypothesis Length) = 1 - WIL

Note: This is the only metric where higher values indicate better performance.

Understanding the Visualizations

Performance Overview

Interactive scatter plot showing the trade-off between processing time (X-axis) and error rate (Y-axis). Points closer to the bottom-left corner represent faster and more accurate transcriptions.

Metrics Heatmap

Color-coded matrix comparing all metrics across configurations. Green indicates good performance, red indicates poor performance. WIP column is inverted since higher values are better.

Box Plots (Resilience Analysis)

Shows the distribution of metric values grouped by engine, degradation type, or other factors. Useful for identifying which engines are most resilient to audio degradation.

  • Box: Shows the interquartile range (IQR) - middle 50% of values
  • Line in box: Median value
  • Whiskers: Extend to 1.5× IQR from the box
  • Points beyond whiskers: Outliers

Transcription Analysis

Side-by-side comparison of reference and hypothesis texts with diff highlighting:

  • Deleted text - Words/characters in reference but missing in hypothesis
  • Inserted text - Words/characters in hypothesis but not in reference

Filters

Use the dropdown filters at the top to focus on specific subsets of results. All tabs respect the current filter selection. Click "Reset" to restore all filters to default.

Metric Normalization

The normalization checkboxes control how metrics (WER, CER, MER, WIL, WIP) are computed:

  • Lowercase: Case-insensitive comparison ("Hello" = "hello")
  • Remove Punctuation: Ignores punctuation marks ("don't" = "dont")
  • Expand Contractions: English only - converts "don't" → "do not"

Note: Text normalization is now a grid search dimension - each combination produces separate results with normalized text shown in the diff view.