Spaces:
Running on Zero
Running on Zero
File size: 5,523 Bytes
5d2d720 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 | # Warbler CDA Test Suite
Comprehensive test suite for the Warbler CDA (Cognitive Development Architecture) RAG system with GPU-accelerated embeddings and FractalStat hybrid scoring.
## Test Organization
### Test Files
1. **test_embedding_providers.py** - Embedding provider tests
- `TestEmbeddingProviderFactory` - Factory pattern tests
- `TestLocalEmbeddingProvider` - Local TF-IDF provider tests
- `TestSentenceTransformerProvider` - GPU-accelerated SentenceTransformer provider tests
- `TestEmbeddingProviderInterface` - Interface contract validation
2. **test_retrieval_api.py** - Retrieval API tests
- `TestRetrievalAPIContextStore` - Document store operations
- `TestRetrievalQueryExecution` - Query execution and filtering
- `TestRetrievalModes` - Different retrieval modes (semantic, temporal, composite)
- `TestRetrievalHybridScoring` - FractalStat hybrid scoring
- `TestRetrievalMetrics` - Metrics and caching
3. **test_fractalstat_integration.py** - FractalStat integration tests
- `TestFractalStatCoordinateComputation` - FractalStat coordinate computation from embeddings
- `TestFractalStatHybridScoring` - Hybrid semantic + FractalStat scoring
- `TestFractalStatDocumentEnrichment` - Document enrichment with FractalStat data
- `TestFractalStatQueryAddressing` - Multi-dimensional query addressing
- `TestFractalStatDimensions` - FractalStat dimensional space properties
4. **test_rag_e2e.py** - End-to-end RAG integration
- `TestEndToEndRAG` - Complete RAG pipeline validation
- 10 comprehensive end-to-end tests covering the full system
## Running Tests
### Install Dependencies
```bash
pip install -r requirements.txt
pip install pytest pytest-cov
```
### Run All Tests
```bash
pytest tests/ -v
```
### Run Specific Test Categories
```bash
# Embedding provider tests
pytest tests/test_embedding_providers.py -v
# Retrieval API tests
pytest tests/test_retrieval_api.py -v
# FractalStat integration tests
pytest tests/test_fractalstat_integration.py -v
# End-to-end tests
pytest tests/test_rag_e2e.py -v -s
```
### Run Tests by Marker
```bash
# Embedding tests
pytest tests/ -m embedding -v
# Retrieval tests
pytest tests/ -m retrieval -v
# FractalStat tests
pytest tests/ -m fractalstat -v
# End-to-end tests
pytest tests/ -m e2e -v -s
# Exclude slow tests
pytest tests/ -m "not slow" -v
```
### Run with Coverage
```bash
pytest tests/ --cov=warbler_cda --cov-report=html -v
```
### Run Specific Test
```bash
pytest tests/test_embedding_providers.py::TestSentenceTransformerProvider::test_semantic_search -v
```
## Test Coverage
The test suite covers:
- ✅ Embedding provider creation and configuration
- ✅ Single text and batch embedding generation
- ✅ Embedding similarity and cosine distance calculations
- ✅ Semantic search across embedding collections
- ✅ Document ingestion into context store
- ✅ Semantic similarity retrieval
- ✅ Temporal sequence retrieval
- ✅ Query result filtering by confidence threshold
- ✅ FractalStat coordinate computation from embeddings
- ✅ FractalStat resonance calculation between documents and queries
- ✅ Hybrid semantic + FractalStat scoring
- ✅ Document enrichment with embeddings and FractalStat data
- ✅ Query result caching and metrics tracking
- ✅ End-to-end RAG pipeline execution
## Dependencies
- **Core**: pytest, warbler-cda
- **Optional**: sentence-transformers (for GPU-accelerated embeddings)
## Expected Test Results
### With SentenceTransformer Installed
All tests pass, including:
- GPU acceleration tests (falls back to CPU if CUDA unavailable)
- FractalStat coordinate computation tests
- Hybrid scoring tests
### Without SentenceTransformer
Tests gracefully skip SentenceTransformer-specific tests and fall back to local TF-IDF provider.
## Writing New Tests
When adding new tests, follow this pattern:
```python
import pytest
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent.parent))
from warbler_cda import RetrievalAPI, RetrievalQuery, RetrievalMode
class TestMyFeature:
"""Test description."""
def setup_method(self):
"""Setup for each test."""
self.api = RetrievalAPI()
def test_my_feature(self):
"""Test my feature."""
# Arrange
self.api.add_document("doc_1", "test")
# Act
result = self.api.retrieve_context(query)
# Assert
assert result is not None
```
## CI/CD Integration
The test suite is designed to work with CI/CD pipelines:
```yaml
# Example GitHub Actions
- name: Run Warbler CDA Tests
run: pytest tests/ --cov=warbler_cda --cov-report=xml
```
## Performance Considerations
- Embedding generation tests are fastest with local TF-IDF provider
- SentenceTransformer tests are slower but more accurate
- First SentenceTransformer test loads the model (cache warmup)
- Subsequent tests benefit from model caching
## Troubleshooting
### ImportError: No module named 'sentence_transformers'
Install the optional dependency:
```bash
pip install sentence-transformers
```
### Tests hang on first SentenceTransformer test
The model is being downloaded. This is normal on first run. Progress can be monitored.
### CUDA out of memory errors
The system automatically falls back to CPU. Tests will still pass but run slower.
### Test file not found
Ensure you're running pytest from the warbler-cda-package directory:
```bash
cd warbler-cda-package
pytest tests/ -v
```
|