Spaces:
Running
Running
| # NLProxy Core Module Reference | |
| This document surveys the core library modules in `core/`. | |
| ## Purpose | |
| Core modules implement the prompt compression pipeline, constraint enforcement, and verification logic used by NLProxy. | |
| ## Files and Responsibilities | |
| ### `core/compressor.py` | |
| #### Purpose | |
| Performs clustering-based sentence compression to reduce prompt size while preserving core meaning. | |
| #### Primary Class | |
| - `SemanticCompressor` | |
| #### Key Concepts | |
| - Uses sentence embeddings to identify semantic redundancy. | |
| - Employs `KMeans` clustering with `n_init="auto"` and `max_iter=300`. | |
| - Returns compressed sentence subsets and compression metrics. | |
| #### Performance | |
| - Complexity: O(n · k · d) for clustering, where n = sentence count, k = cluster count, d = embedding dimension. | |
| - Best suited to sentence sizes under 100 to avoid quadratic cluster overhead. | |
| #### Edge Cases | |
| - Handles empty sentence lists as a no-op. | |
| - Falls back to conservative compression when cluster quality is low. | |
| ### `core/corrector.py` | |
| #### Purpose | |
| Sanitizes and post-corrects LLM outputs to align with safety constraints extracted during prompt shielding. | |
| #### Primary Class | |
| - `ResponseCorrector` | |
| #### Behavior | |
| - Applies heuristic cleanup to generated text. | |
| - Mends broken placeholders and ensures protected entity tokens are restored correctly. | |
| #### Edge Cases | |
| - Responds defensively when correction data is missing. | |
| - Avoids overcorrection by preserving valid syntax. | |
| ### `core/model_manager.py` | |
| #### Purpose | |
| Coordinates local model verification, SHA256 checksum validation, and on-demand download triggers. | |
| #### Primary Classes | |
| - `ModelConfig` | |
| - `ModelManager` | |
| #### Features | |
| - Thread-safe singleton initialization. | |
| - `verify_zip_checksum(zip_path, expected_hash)` validates download integrity. | |
| - `ensure_ready()` is async-safe and idempotent. | |
| - Supports synchronous initialization via `sync_ensure_ready()`. | |
| #### Edge Cases | |
| - Raises `RuntimeError` if required models remain missing after download. | |
| - Uses atomic file moves to avoid partial ZIP writes. | |
| ### `core/reconstructor.py` | |
| #### Purpose | |
| Reconstructs compressed prompts by reinserting protected entities, formatting, and token optimization artifacts. | |
| #### Primary Classes | |
| - `ReconstructionResult` | |
| - `PromptReconstructor` | |
| #### Behavior | |
| - Rebuilds final prompt text from compressed sentences and placeholder maps. | |
| - Produces token counts and compression metrics. | |
| #### Complexity | |
| - Reconstruction is O(n) in the number of sentences and placeholder replacements. | |
| ### `core/restriction.py` | |
| #### Purpose | |
| Represents extracted restrictions, blocklists, and prompt constraints during shielding. | |
| #### Primary Classes | |
| - `Restriction` | |
| - `RestrictionGraph` | |
| #### Behavior | |
| - Encodes restriction rules as immutable dataclasses. | |
| - Builds a directed graph of dependents for conflict detection. | |
| #### Edge Cases | |
| - Handles conflicting restrictions with explicit priority logic. | |
| ### `core/safety.py` | |
| #### Purpose | |
| Validates compressed prompts and generated responses against safety policies. | |
| #### Primary Classes | |
| - `SafetyReport` | |
| - `SafetyChecker` | |
| #### Features | |
| - Enforces semantic drift thresholds. | |
| - Optional perplexity-based checks. | |
| - Supports multiple safety modes. | |
| #### Performance | |
| - Safety validation is lightweight compared to embedding inference. | |
| - Perplexity checks are conditional and only activate when enabled. | |
| ### `core/segmenter.py` | |
| #### Purpose | |
| Segments text into sentences and generates dense embeddings. | |
| #### Primary Classes | |
| - `EmbeddingBackend` | |
| - `SegmentationConfig` | |
| - `SemanticSegmenter` | |
| #### Features | |
| - Supports ONNX and PyTorch backends. | |
| - Uses local model artifacts from `nlproxy/models/`. | |
| - Selects CPU INT8 ONNX models when `onnx_int8=True`. | |
| - Exposes async inference via `segment_and_encode_async()`. | |
| #### Performance | |
| - Sentence segmentation is O(n) in prompt length. | |
| - Embedding inference latency depends on backend and hardware. | |
| - Recommended batch sizes: `32-128` for CPU production. | |
| #### Edge Cases | |
| - Fails fast when local model files are missing. | |
| - Uses `NLPROXY_MODELS_DIR` environment variable for model path override. | |
| ### `core/shield.py` | |
| #### Purpose | |
| Protects sensitive content, extracts protected entities, and applies domain-specific shielding. | |
| #### Primary Classes | |
| - `DomainMode` | |
| - `ProtectedBlock` | |
| - `ProtectedEntity` | |
| - `ShieldResult` | |
| - `PromptShield` | |
| #### Behavior | |
| - Detects password-like tokens, email addresses, and other protected spans. | |
| - Replaces protected text with deterministic placeholders. | |
| - Produces `ShieldResult` with `placeholder_map` and `shielded_text`. | |
| #### Edge Cases | |
| - Ensures placeholder collision resistance using secrets and hashing. | |
| - Maintains alignment for re-injection during reconstruction. | |
| ### `core/verifier.py` | |
| #### Purpose | |
| Performs post-LLM verification and hallucination detection. | |
| #### Primary Classes | |
| - `VerificationResult` | |
| - `PostLLMVerifier` | |
| #### Behavior | |
| - Checks final response confidence scores. | |
| - Reports violations and corrective action recommendations. | |
| - Integrates with automatic correction loops. | |
| #### Edge Cases | |
| - Gracefully disables NLI verification if models are unavailable. | |
| - Allows response acceptance only when `confidence_score` exceeds configured thresholds. | |