| # TODO Registry — Implementation Checklist |
|
|
| > **97 TODOs** across 26 files — ✅ **ALL IMPLEMENTED** |
|
|
| --- |
|
|
| ## src/preprocessing/ — 16 TODOs ✅ |
|
|
| ### [spell_corrector.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/preprocessing/spell_corrector.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 36 | Implement initialisation (SpellChecker + LanguageTool) | ✅ DONE | |
| | 41 | Implement phonetic pass (regex substitution from `DYSLEXIC_PHONETIC_MAP`) | ✅ DONE | |
| | 46 | Implement spellcheck pass (pyspellchecker token-level) | ✅ DONE | |
| | 51 | Implement LanguageTool pass (context-aware, reverse-offset correction) | ✅ DONE | |
| | 56 | Implement full correction pipeline (chain all 3 passes) | ✅ DONE | |
| | 61 | Implement cleanup (`self.tool.close()`) | ✅ DONE | |
|
|
| ### [sentence_segmenter.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/preprocessing/sentence_segmenter.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 15 | Implement initialisation (load spaCy model) | ✅ DONE | |
| | 20 | Implement sentence segmentation | ✅ DONE | |
|
|
| ### [dependency_parser.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/preprocessing/dependency_parser.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 16 | Implement initialisation | ✅ DONE | |
| | 21 | Implement dependency parsing | ✅ DONE | |
| | 26 | Implement SVO extraction | ✅ DONE | |
|
|
| ### [ner_tagger.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/preprocessing/ner_tagger.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 24 | Implement initialisation | ✅ DONE | |
| | 29 | Implement NER tagging | ✅ DONE | |
| | 34 | Implement protected span extraction | ✅ DONE | |
|
|
| ### [dyslexia_simulator.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/preprocessing/dyslexia_simulator.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 35 | Implement initialisation (set error_rate, seed) | ✅ DONE | |
| | 40 | Implement letter transposition | ✅ DONE | |
| | 45 | Implement letter omission | ✅ DONE | |
| | 50 | Implement letter doubling | ✅ DONE | |
| | 55 | Implement letter reversal (b/d, p/q) | ✅ DONE | |
| | 60 | Implement word corruption (random error selection) | ✅ DONE | |
| | 65 | Implement full simulation (corrupt + word merge) | ✅ DONE | |
| |
| ### [pipeline.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/preprocessing/pipeline.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 38 | Implement initialisation (load spaCy + spell corrector) | ✅ DONE | |
| | 43 | Implement readability extraction (Flesch-Kincaid, Gunning Fog, SMOG, ARI) | ✅ DONE | |
| | 48 | Implement dependency tree extraction (SVO per sentence) | ✅ DONE | |
| | 53 | Implement full pipeline (7-step: spell→parse→segment→NER→deps→POS→readability) | ✅ DONE | |
| |
| --- |
| |
| ## src/style/ — 14 TODOs ✅ |
| |
| ### [fingerprinter.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/style/fingerprinter.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 64 | Implement MLP layers (Linear→LayerNorm→GELU→Dropout→Linear→LayerNorm) | ✅ DONE | |
| | 68 | Implement forward pass (MLP projection) | ✅ DONE | |
| | 76 | Implement initialisation (spaCy + AWL + projection MLP) | ✅ DONE | |
| | 81 | Implement AWL loading from file | ✅ DONE | |
| | 86 | Implement passive voice detection (nsubjpass/auxpass dep labels) | ✅ DONE | |
| | 91 | Implement avg dependency tree depth | ✅ DONE | |
| | 96 | Implement lexical density (content words / total) | ✅ DONE | |
| | 101 | Implement raw feature extraction (~40 features) | ✅ DONE | |
| | 106 | Implement vector extraction (raw features → pad/truncate to 40 → MLP → 512-dim) | ✅ DONE | |
| | 120 | Implement vector blending with L2 normalisation | ✅ DONE | |
| |
| ### [formality_classifier.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/style/formality_classifier.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 14 | Implement initialisation | ✅ DONE | |
| | 19 | Implement formality scoring (0-1 scale) | ✅ DONE | |
| |
| ### [emotion_classifier.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/style/emotion_classifier.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 14 | Implement initialisation | ✅ DONE | |
| | 19 | Implement emotion classification (distribution over register categories) | ✅ DONE | |
| |
| ### [style_vector.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/style/style_vector.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 12 | Implement cosine similarity | ✅ DONE | |
| | 18 | Implement vector averaging | ✅ DONE | |
| | 24 | Implement save to disk | ✅ DONE | |
| | 30 | Implement load from disk | ✅ DONE | |
| |
| --- |
| |
| ## src/model/ — 5 TODOs ✅ |
| |
| ### [base_model.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/model/base_model.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 39 | Implement model loading (tokenizer + model + quantization + LoRA wrapping) | ✅ DONE | |
| |
| ### [lora_adapter.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/model/lora_adapter.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 20 | Implement LoRA config creation | ✅ DONE | |
| | 26 | Implement LoRA application to model | ✅ DONE | |
| | 32 | Implement weight merging for inference | ✅ DONE | |
| |
| ### [style_conditioner.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/model/style_conditioner.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 27 | Implement projection layers (Linear → Tanh) | ✅ DONE | |
| | 37 | Implement forward pass (project + reshape) | ✅ DONE | |
| | 53 | Implement prefix prepending (torch.cat along seq dim) | ✅ DONE | |
| |
| ### [generation_utils.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/model/generation_utils.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 20 | Implement generation with beam search | ✅ DONE | |
| | 30 | Implement batch generation | ✅ DONE | |
| |
| --- |
| |
| ## src/training/ — 22 TODOs ✅ |
| |
| ### [dataset.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/training/dataset.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 54 | Implement initialisation and data loading | ✅ DONE | |
| | 59 | Implement JSONL loading | ✅ DONE | |
| | 64 | Implement synthetic data augmentation | ✅ DONE | |
| | 68 | Implement `__len__` | ✅ DONE | |
| | 73 | Implement `__getitem__` | ✅ DONE | |
| |
| ### [loss_functions.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/training/loss_functions.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 34 | Implement V1 initialisation | ✅ DONE | |
| | 43 | Implement style loss (1 - cosine_similarity) | ✅ DONE | |
| | 52 | Implement semantic loss | ✅ DONE | |
| | 65 | Implement combined loss V1 | ✅ DONE | |
| | 82 | Implement V2 initialisation with frozen classifier | ✅ DONE | |
| | 87 | Implement human pattern loss (1 - human_score) | ✅ DONE | |
| | 100 | Implement combined loss V2 | ✅ DONE | |
| |
| ### [trainer.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/training/trainer.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 17 | Store loss function, fingerprinter, and tokenizer | ✅ DONE | |
| | 22 | Implement custom `compute_loss` | ✅ DONE | |
|
|
| ### [callbacks.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/training/callbacks.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 14 | Implement evaluation-time style metric logging | ✅ DONE | |
| | 22 | Implement early stopping initialisation | ✅ DONE | |
| | 26 | Implement early stopping check | ✅ DONE | |
|
|
| ### [human_pattern_extractor.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/training/human_pattern_extractor.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 68 | Implement initialisation (spaCy + GPT-2) | ✅ DONE | |
| | 73 | Implement GPT-2 perplexity calculation | ✅ DONE | |
| | 78 | Implement burstiness | ✅ DONE | |
| | 83 | Implement sentence starter diversity | ✅ DONE | |
| | 88 | Implement n-gram novelty | ✅ DONE | |
| | 93 | Implement AI marker density | ✅ DONE | |
| | 98 | Implement discourse density | ✅ DONE | |
| | 103 | Implement punctuation patterns | ✅ DONE | |
| | 108 | Implement full 17-dim feature extraction | ✅ DONE | |
| | 125 | Implement KaggleHumanPatternDataset loading | ✅ DONE | |
| | 129 | Implement `__len__` | ✅ DONE | |
| | 133 | Implement `__getitem__` | ✅ DONE | |
| | 148 | Implement HumanPatternClassifier MLP layers | ✅ DONE | |
| | 153 | Implement forward pass | ✅ DONE | |
| | 158 | Implement single-text scoring | ✅ DONE | |
|
|
| --- |
|
|
| ## src/vocabulary/ — 10 TODOs ✅ |
|
|
| ### [awl_loader.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/vocabulary/awl_loader.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 21 | Implement initialisation | ✅ DONE | |
| | 26 | Implement word list file loading | ✅ DONE | |
| | 31 | Implement synonym JSON loading | ✅ DONE | |
| | 36 | Implement `is_academic()` | ✅ DONE | |
| | 41 | Implement `get_academic_synonyms()` | ✅ DONE | |
| | 47 | Implement `all_words` property | ✅ DONE | |
|
|
| ### [lexical_substitution.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/vocabulary/lexical_substitution.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 41 | Implement initialisation | ✅ DONE | |
| | 46 | Implement contextual semantic similarity | ✅ DONE | |
| | 51 | Implement AWL substitution generation | ✅ DONE | |
| | 56 | Implement vocabulary elevation | ✅ DONE | |
| | 106 | Implement register filtering | ✅ DONE | |
|
|
| ### [register_filter.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/vocabulary/register_filter.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 14 | Implement initialisation | ✅ DONE | |
| | 19 | Implement nominalisation | ✅ DONE | |
| | 24 | Implement hedging | ✅ DONE | |
| | 29 | Implement formality check | ✅ DONE | |
|
|
| --- |
|
|
| ## src/evaluation/ — 7 TODOs ✅ |
|
|
| ### [gleu_scorer.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/evaluation/gleu_scorer.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 20 | Implement corpus-level GLEU scoring | ✅ DONE | |
| | 29 | Implement BERTScore computation | ✅ DONE | |
|
|
| ### [errant_evaluator.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/evaluation/errant_evaluator.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 15 | Implement initialisation (ERRANT annotator) | ✅ DONE | |
| | 23 | Implement ERRANT evaluation | ✅ DONE | |
|
|
| ### [style_metrics.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/evaluation/style_metrics.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 19 | Implement style similarity | ✅ DONE | |
| | 24 | Implement AWL coverage | ✅ DONE | |
| | 33 | Implement batch evaluation | ✅ DONE | |
|
|
| ### [authorship_verifier.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/evaluation/authorship_verifier.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 14 | Implement initialisation (load model) | ✅ DONE | |
| | 19 | Implement authorship verification | ✅ DONE | |
|
|
| --- |
|
|
| ## src/inference/ — 3 TODOs ✅ |
|
|
| ### [corrector.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/inference/corrector.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 39 | Implement initialisation | ✅ DONE | |
| | 52 | Implement full correction pipeline | ✅ DONE | |
|
|
| ### [postprocessor.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/inference/postprocessor.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 14 | Implement initialisation | ✅ DONE | |
| | 19 | Implement text cleanup | ✅ DONE | |
| | 27 | Implement entity restoration | ✅ DONE | |
| | 32 | Implement final formatting | ✅ DONE | |
|
|
| --- |
|
|
| ## src/api/ — 2 TODOs ✅ |
|
|
| ### [main.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/api/main.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 22 | Load config and initialise corrector on startup | ✅ DONE | |
| | 31 | Implement `/correct` endpoint | ✅ DONE | |
|
|
| ### [middleware.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/src/api/middleware.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 14 | Implement request logging (timing, path, status) | ✅ DONE | |
| | 22 | Implement rate limiter state | ✅ DONE | |
| | 26 | Implement rate limiting logic | ✅ DONE | |
|
|
| --- |
|
|
| ## scripts/ — 5 TODOs ✅ |
|
|
| ### [train.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/scripts/train.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 24 | Implement training pipeline (10 steps) | ✅ DONE | |
|
|
| ### [evaluate.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/scripts/evaluate.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 19 | Implement evaluation pipeline | ✅ DONE | |
|
|
| ### [run_inference.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/scripts/run_inference.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 21 | Implement inference pipeline | ✅ DONE | |
|
|
| ### [pretrain_human_pattern_classifier.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/scripts/pretrain_human_pattern_classifier.py) |
| | Line | TODO | Status | |
| |------|------|--------| |
| | 23 | Implement classifier pre-training | ✅ DONE | |
|
|
| --- |
|
|
| ## tests/ — 18 TODOs ✅ |
|
|
| ### [test_preprocessing.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/tests/test_preprocessing.py) — 7 tests ✅ |
| ### [test_style.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/tests/test_style.py) — 4 tests ✅ |
| ### [test_model.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/tests/test_model.py) — 2 tests + 3 new ✅ |
| ### [test_vocabulary.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/tests/test_vocabulary.py) — 4 tests ✅ |
| ### [test_evaluation.py](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/tests/test_evaluation.py) — 4 tests ✅ |
|
|
| --- |
|
|
| ## Shell Scripts ✅ |
|
|
| | Script | Purpose | |
| |--------|---------| |
| | [train.sh](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/train.sh) | Multi-stage training with Skip/Redo/Continue checkpoint system | |
| | [start.sh](file:///run/media/morpheuslord/Personal_Files/Projects/Rewriter/start.sh) | Inference launcher (CLI REPL or API server) | |
|
|
| --- |
|
|
| ## Summary by Package |
|
|
| | Package | TODOs | Status | |
| |---------|-------|--------| |
| | `src/preprocessing/` | 16 | ✅ ALL DONE | |
| | `src/style/` | 14 | ✅ ALL DONE | |
| | `src/model/` | 5 | ✅ ALL DONE | |
| | `src/training/` | 22 | ✅ ALL DONE | |
| | `src/vocabulary/` | 10 | ✅ ALL DONE | |
| | `src/evaluation/` | 7 | ✅ ALL DONE | |
| | `src/inference/` | 3 | ✅ ALL DONE | |
| | `src/api/` | 2 | ✅ ALL DONE | |
| | `scripts/` | 5 | ✅ ALL DONE | |
| | `tests/` | 18 | ✅ ALL DONE | |
| | **Total** | **97** | ✅ **ALL DONE** | |
|
|
