Spaces:
Running
Running
Update guidelines.md
Browse files- guidelines.md +9 -9
guidelines.md
CHANGED
|
@@ -6,11 +6,11 @@ The evaluation focuses exclusively on language quality, naturalness, and grammat
|
|
| 6 |
|
| 7 |
# Definitions
|
| 8 |
|
| 9 |
-
### What is fluency?
|
| 10 |
|
| 11 |
Fluency refers to the linguistic quality of text that makes it natural, smooth, and easy to read. It should look like a text written by a native speaker. A fluent text should consistently use either Bokmål or Nynorsk (depending on the prompt), and should sound genuinely Norwegian rather than as it were translated from another language.
|
| 12 |
|
| 13 |
-
### Fluency issues to look for
|
| 14 |
|
| 15 |
When evaluating fluency, pay attention to:
|
| 16 |
|
|
@@ -24,7 +24,7 @@ When evaluating fluency, pay attention to:
|
|
| 24 |
|
| 25 |
# Annotation procedure
|
| 26 |
|
| 27 |
-
### Step-by-Step process
|
| 28 |
|
| 29 |
1. **Read the prompt**: Do not analyze the fluency of the prompt, but look at it to understand the context and language style.
|
| 30 |
2. **Read both responses completely** without making immediate judgments
|
|
@@ -32,7 +32,7 @@ When evaluating fluency, pay attention to:
|
|
| 32 |
4. **Compare the severity and frequency** of fluency issues between responses
|
| 33 |
5. **Make your decision** based on overall fluency
|
| 34 |
|
| 35 |
-
### Decision options
|
| 36 |
|
| 37 |
You must select one of three options:
|
| 38 |
|
|
@@ -40,7 +40,7 @@ You must select one of three options:
|
|
| 40 |
- **B is more fluent**: Response B has better overall language quality than Response A
|
| 41 |
- **Equally fluent**: Both responses have similar language quality (minor differences that don't clearly favor either response)
|
| 42 |
|
| 43 |
-
### Important guidelines
|
| 44 |
|
| 45 |
- **Minor differences matter**: Even small improvements in fluency should influence your decision
|
| 46 |
- **Be consistent**: Apply the same standards across all evaluations
|
|
@@ -48,19 +48,19 @@ You must select one of three options:
|
|
| 48 |
|
| 49 |
# Examples
|
| 50 |
|
| 51 |
-
### Example 1: Clear fluency difference
|
| 52 |
|
| 53 |
TODO
|
| 54 |
|
| 55 |
-
### Example 2: Equal fluency
|
| 56 |
|
| 57 |
TODO
|
| 58 |
|
| 59 |
-
### Example 3: Subtle fluency difference
|
| 60 |
|
| 61 |
TODO
|
| 62 |
|
| 63 |
-
### Example 4: Content vs. fluency
|
| 64 |
|
| 65 |
TODO
|
| 66 |
|
|
|
|
| 6 |
|
| 7 |
# Definitions
|
| 8 |
|
| 9 |
+
#### What is fluency?
|
| 10 |
|
| 11 |
Fluency refers to the linguistic quality of text that makes it natural, smooth, and easy to read. It should look like a text written by a native speaker. A fluent text should consistently use either Bokmål or Nynorsk (depending on the prompt), and should sound genuinely Norwegian rather than as it were translated from another language.
|
| 12 |
|
| 13 |
+
#### Fluency issues to look for
|
| 14 |
|
| 15 |
When evaluating fluency, pay attention to:
|
| 16 |
|
|
|
|
| 24 |
|
| 25 |
# Annotation procedure
|
| 26 |
|
| 27 |
+
#### Step-by-Step process
|
| 28 |
|
| 29 |
1. **Read the prompt**: Do not analyze the fluency of the prompt, but look at it to understand the context and language style.
|
| 30 |
2. **Read both responses completely** without making immediate judgments
|
|
|
|
| 32 |
4. **Compare the severity and frequency** of fluency issues between responses
|
| 33 |
5. **Make your decision** based on overall fluency
|
| 34 |
|
| 35 |
+
#### Decision options
|
| 36 |
|
| 37 |
You must select one of three options:
|
| 38 |
|
|
|
|
| 40 |
- **B is more fluent**: Response B has better overall language quality than Response A
|
| 41 |
- **Equally fluent**: Both responses have similar language quality (minor differences that don't clearly favor either response)
|
| 42 |
|
| 43 |
+
#### Important guidelines
|
| 44 |
|
| 45 |
- **Minor differences matter**: Even small improvements in fluency should influence your decision
|
| 46 |
- **Be consistent**: Apply the same standards across all evaluations
|
|
|
|
| 48 |
|
| 49 |
# Examples
|
| 50 |
|
| 51 |
+
#### Example 1: Clear fluency difference
|
| 52 |
|
| 53 |
TODO
|
| 54 |
|
| 55 |
+
#### Example 2: Equal fluency
|
| 56 |
|
| 57 |
TODO
|
| 58 |
|
| 59 |
+
#### Example 3: Subtle fluency difference
|
| 60 |
|
| 61 |
TODO
|
| 62 |
|
| 63 |
+
#### Example 4: Content vs. fluency
|
| 64 |
|
| 65 |
TODO
|
| 66 |
|