File size: 4,180 Bytes
2a64ad4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 |
# Commitment Conservation Test Harness
This directory contains the test harness for validating the commitment conservation law under compression and recursive application.
## Overview
The harness implements a falsification framework that operationalizes commitment invariance using:
- Compression-based stress tests
- Lineage-aware evaluation
- Model-agnostic testing infrastructure
## Structure
- **src/** - Core harness implementation
- `harness.py` - Main test harness
- `test_harness.py` - Test framework
- `extraction.py` - Commitment extraction
- `metrics.py` - Evaluation metrics
- `samples.py` - Sample data management
- `plotting.py` - Visualization utilities
- `config.py` - Configuration management
- `deterministic_pipeline.py` - Deterministic testing pipeline
- `advanced_extractor.py` - Advanced extraction methods
- **tests/** - Unit and integration tests
- `test_harness.py` - Harness tests
- `test_full_harness.py` - Full integration tests
## Configuration Files
- `requirements.txt` - Python dependencies
- `pyproject.toml` - Project configuration
- `environment.yml` - Conda environment specification
- `Harnesstest.ini` - Harness configuration
## Quick Start
```bash
# Install dependencies
pip install -r requirements.txt
python -m spacy download en_core_web_sm
# Extract commitments from text
python analyze.py "You must complete this by Friday."
# Run experiments (compression/recursion tests)
python analyze.py run compression --signal "You must complete this by Friday."
python analyze.py run recursion --signal "Contract terms apply." --depth 5
```
### Installation
```bash
# Install dependencies
pip install -r requirements.txt
# Download spaCy model
python -m spacy download en_core_web_sm
```
### CLI Usage
**Quick test** — Extract commitments from text:
```bash
python analyze.py "You must complete this by Friday."
python analyze.py "Contract terms apply." --quiet
python analyze.py "You shall deliver by Monday." --json
```
**Experiments** — Run compression/recursion tests and generate receipts:
```bash
python analyze.py run compression --signal "You must complete this by Friday."
python analyze.py run recursion --signal "You must pay $100." --depth 5
python analyze.py run full
```
Experimental runs output timestamped JSON receipts to `outputs/` directory.
### Running Tests
```bash
# Run all tests
MPLBACKEND=Agg pytest tests/test_full_harness.py -v
# Quick test run
MPLBACKEND=Agg pytest tests/test_full_harness.py -q
```
## CLI Commands
### `compression`
Tests commitment conservation under compression transformations.
Options:
- `--signal TEXT` - Input signal text (required)
- `--out PATH` - Output receipt path (default: `outputs/compression_receipt.json`)
### `recursion`
Tests commitment drift under recursive transformations.
Options:
- `--signal TEXT` - Input signal text (required)
- `--depth N` - Recursion depth (default: 8)
- `--enforced` - Enable enforcement mode
- `--out PATH` - Output receipt path (default: `outputs/recursion_receipt.json`)
### `full`
Runs the complete deterministic pipeline.
Options:
- `--out PATH` - Output receipt path (default: `outputs/full_receipt.json`)
## Output
The CLI generates:
- JSON receipts in `outputs/` with all experimental data
- Compression fidelity plots (`fid_*.png`)
- Recursion drift plots (`delta_*.png`)
## Purpose
This harness is designed to test whether commitment content is conserved under:
1. **Compression** - Reduction to essential structure
2. **Recursion** - Repeated self-application with lineage tracking
The framework is model-agnostic and can be applied to both human and machine-generated language.
### Environment Notes
This harness specifies the structure and invariants of the evaluation.
Exact dependency resolution may vary across systems (OS, Python, backend).
Environment-related failures should be distinguished from invariant violations.
## Citation
If you use this harness, please cite the original paper:
```text
McHenry, D. J. (2026). A Conservation Law for Commitment in Language Under
Transformative Compression and Recursive Application. Zenodo.
DOI: 10.5281/zenodo.18267279
```
|