Download docs/PROPERTY_TESTING.md from veripazari/durak: direct link, hf CLI and curl.
- Browser
- Download file 7.07 kB
-
https://huggingface.co/veripazari/durak/resolve/main/docs/PROPERTY_TESTING.md
- Command line
-
hf download hf://veripazari/durak/docs/PROPERTY_TESTING.md
-
curl -L -o PROPERTY_TESTING.md https://huggingface.co/veripazari/durak/resolve/main/docs/PROPERTY_TESTING.md
Property-Based Testing Guide
Durak uses Hypothesis for property-based testing to ensure robustness across a wide range of Turkish text inputs.
What is Property-Based Testing?
Unlike traditional unit tests that check specific examples, property-based tests verify mathematical properties that should hold for all inputs.
Example: Traditional vs Property-Based
Traditional test:
def test_normalize_case():
assert normalize_case("İSTANBUL") == "istanbul"
assert normalize_case("ANKARA") == "ankara"
Property-based test:
@given(turkish_word())
def test_normalize_case_is_idempotent(word):
"""Normalizing twice should equal normalizing once."""
normalized_once = normalize_case(word)
normalized_twice = normalize_case(normalized_once)
assert normalized_once == normalized_twice
The property test generates hundreds of random Turkish words and verifies the idempotence property holds for all of them.
Why Property-Based Testing for Turkish NLP?
Turkish has unique challenges that make property testing especially valuable:
- Diacritic complexity: İ/I vs i/ı conversion must be correct in all cases
- Apostrophe handling:
İstanbul'a,Ankara'dacreate complex token boundaries - Unicode edge cases: Zero-width characters, combining diacritics, RTL marks
- Morphological richness: Agglutination creates extremely long, unpredictable words
- Capitalization rules: Turkish-specific uppercase/lowercase conversions
Traditional unit tests can't cover the combinatorial space of these features.
Text Generation Strategies
Durak provides specialized Hypothesis strategies in tests/strategies.py:
Basic Strategies
from tests.strategies import turkish_word, turkish_sentence
# Generate random Turkish words
turkish_word() # "çalışmak", "görmek", "İstanbul", etc.
# Generate words with possessive/case markers
turkish_word_with_suffix() # "İstanbul'a", "Ankara'dan", etc.
# Generate full sentences
turkish_sentence() # "İstanbul'a gittik ve müze'yi gördük."
Advanced Strategies
# Unicode edge cases (zero-width chars, combining diacritics)
turkish_text_with_unicode_edge_cases()
# Lists of Turkish stopwords
turkish_stopword_list()
Writing Property Tests
1. Import the Strategy
from hypothesis import given
from tests.strategies import turkish_sentence
2. Define the Property
@given(turkish_sentence())
def test_tokenize_always_returns_list(text):
"""Tokenization must always return a list."""
tokens = tokenize(text)
assert isinstance(tokens, list)
3. Run with Hypothesis
Hypothesis will:
- Generate 100-200 random examples (configurable with
@settings(max_examples=N)) - Run your test on each example
- If a failure occurs, shrink to find the minimal failing case
- Cache failures for regression testing
Common Properties to Test
Idempotence
Functions that should give the same result when applied twice:
@given(turkish_sentence())
def test_normalize_is_idempotent(text):
assert normalize_case(text) == normalize_case(normalize_case(text))
Inverse Functions
Functions that should undo each other:
@given(turkish_sentence())
def test_tokenize_rejoins(text):
tokens = tokenize(text)
rejoined = "".join(tokens)
# Should preserve non-whitespace content
assert set(rejoined.replace(" ", "")) == set(text.replace(" ", ""))
Monotonicity
Functions that should never increase size:
@given(turkish_word())
def test_normalization_not_expanding(word):
assert len(normalize_case(word)) <= len(word)
Consistency
Related functions should agree:
@given(turkish_sentence())
def test_offset_consistency(text):
tokens_with_offsets = tokenize_with_offsets(text)
for token, start, end in tokens_with_offsets:
# Offset must be valid
assert 0 <= start < end <= len(text)
# Extracted text should match token
extracted = text[start:end]
assert extracted.lower() == token.lower()
Controlling Test Execution
Number of Examples
from hypothesis import settings
@given(turkish_sentence())
@settings(max_examples=500) # Generate 500 random examples
def test_something(text):
...
Filtering Invalid Inputs
from hypothesis import assume
@given(turkish_sentence())
def test_non_empty_tokenization(text):
assume(len(text.strip()) > 0) # Skip empty strings
tokens = tokenize(text)
assert len(tokens) > 0
Debugging Failures
When a property test fails, Hypothesis prints:
- The minimal failing example (after shrinking)
- The random seed to reproduce the failure
Falsifying example: test_something(text='İ')
You can reproduce this by running:
pytest --hypothesis-seed=12345
Running Property Tests
Run all property tests:
pytest tests/test_properties.py -v
Show Hypothesis statistics:
pytest tests/test_properties.py --hypothesis-show-statistics
Output:
- 200 passing examples, 0 failing examples, 0 invalid examples
- Typical runtimes: 0-1ms, ~ 50% in data generation
- Stopped because settings.max_examples=200
Run with more examples (stress test):
pytest tests/test_properties.py --hypothesis-profile=ci
Best Practices
✅ Do:
- Test mathematical properties, not specific examples
- Use
assume()to filter out invalid inputs - Keep properties simple and focused (one property per test)
- Use meaningful assertion messages
- Combine traditional and property-based tests
❌ Don't:
- Test implementation details
- Make tests too complex (they become hard to debug)
- Over-constrain input generation (reduces coverage)
- Ignore failures (Hypothesis finds real edge cases!)
Example: Real Bug Caught by Property Testing
Property test:
@given(turkish_word())
def test_normalize_case_removes_uppercase(word):
normalized = normalize_case(word)
assert "İ" not in normalized
Bug found:
Falsifying example: test_normalize_case_removes_uppercase(word='İİİ')
AssertionError: 'İ' found in normalized output
Root cause: Normalization was using Python's .lower() instead of Turkish-aware lowercase conversion. Property testing caught it instantly!
Resources
Contributing
When adding new features to Durak:
- Add traditional unit tests for specific examples
- Add property tests for general behavior
- Document invariants that your function guarantees
Property tests serve as executable specifications and protect against regressions.