Buckets:
| # SSK_Hardware Architecture | |
| > [!IMPORTANT] | |
| > **Version Control and State Tracking Rule**: | |
| > This repository exclusively uses **`lore`** for version control, state management, and documentation tracking. **Do NOT use `git`**. | |
| --- | |
| ## 1. Overview and Purpose | |
| `SSK_Hardware` is the hardware-accelerated audio processing engine of the **SovereignSurvivalKit** architecture. It integrates Apple Silicon machine learning execution (via `SSK_Demucs_MLX`), vector DSP acceleration (via Apple `Accelerate` `vDSP`), and system-level musical feature analysis (via Apple `MusicUnderstanding`). | |
| The module performs: | |
| 1. Real-time audio frame telemetry generation (calculating $\tau$-dynamics and Maslow domain projections). | |
| 2. Neural 4-stem audio source separation (Vocals, Drums, Bass, Other). | |
| 3. Pre-pass macro rhythm, pace, and structural section analysis. | |
| --- | |
| ## 2. Directory Structure and Files | |
| ```text | |
| SSK_Hardware/ | |
| ├── Package.swift | |
| ├── ARCHITECTURE.md | |
| └── Sources/ | |
| └── SSK_Hardware/ | |
| ├── HardwareAudioPipeline.swift # Async audio analysis protocol contract | |
| ├── AudioTelemetryPipeline.swift # Central 11-dimension tau-dynamics pipeline | |
| ├── MLXDemucsPipeline.swift # High-throughput MLX neural stem separation | |
| ├── DemucsSeparator.swift # Fallback CoreML FP16 stem separator | |
| ├── MusicUnderstandingExtractor.swift # Apple MusicUnderstanding pre-processor | |
| └── HardwareConfiguration.swift # Runtime hardware engine configuration | |
| ``` | |
| --- | |
| ## 3. Subsystem Architecture | |
| ### 1. `HardwareAudioPipeline.swift` | |
| Defines the base protocol contract for streaming audio telemetry frames: | |
| - `func analyzeFile(fileURL: URL) async throws -> AsyncThrowingStream<(HierarchicalAudioFrame, [Double]), Error>` | |
| ### 2. `AudioTelemetryPipeline.swift` | |
| Implements `AudioTelemetryPipelineProtocol`: | |
| - **$\tau$-Dynamics Calculation**: Computes real-time velocity ($\Delta X / \Delta T$) across 11 structural gap dimensions using `vDSP_vsubD` and `vDSP_vdivD`. | |
| - **Amodal Inversion Routing**: Forwards structural gap measurements directly to `AmodalInversionEngine` (from `SSK_Physics`). | |
| - **Tier 1 (Chromagram & Dissonance)**: Computes 12-bin normalized chromagram using forward FFT (`vDSP_fft_zrip`). | |
| - **Tier 2 (Stem Separation)**: Delegates neural stem separation to `MLXDemucsPipeline` or `DemucsSeparator`. | |
| - **Tier 4 (Signature Subtraction)**: Computes phase-cancellation over aligned audio offset windows. | |
| - **Adaptive PCA Projection**: Projects 11 gap dimensions onto an 8-domain Maslow hierarchy using dynamic $P_0$ matrices and covariance updates (`updatePCA`). | |
| ### 3. `MLXDemucsPipeline.swift` & `VDSPExtractor` | |
| High-performance Apple Silicon MLX audio stem separator: | |
| 1. Loads model weights via `WeightLoader` (from `SSK_Demucs_MLX`). | |
| 2. Runs pre-pass `MusicUnderstandingExtractor` analysis for rhythm, pace, and section markers. | |
| 3. Slices 10-second audio chunks (441,000 samples at 44.1 kHz) through `DemucsModel` on MLX. | |
| 4. Distributes telemetry extraction across worker tasks in a `TaskGroup` over 8,192-sample subframes with 50% overlap. | |
| 5. Encrypts extracted stem audio files at rest using AES-GCM via `VaultKeyManager`. | |
| ### 4. `DemucsSeparator.swift` | |
| Fallback CoreML stem separator using `HTDemucs_CoreML_FP16.mlpackage`: | |
| - Implements triangular window overlap-add chunk processing. | |
| - Separates audio into `.vocals`, `.drums`, `.bass`, and `.other` buffers. | |
| ### 5. `MusicUnderstandingExtractor.swift` | |
| Pre-processes audio tracks using Apple's `MusicUnderstanding` system framework: | |
| - Evaluates macro-level rhythm, tempo/pace, and structural section boundaries. | |
| - Provides binary search lookups over beat timetables (`CMTime`) and section ranges (`CMTimeRange`). | |
| --- | |
| ## 4. Module Dependencies | |
| ### Internal Package Dependencies (`Package.swift`) | |
| - **`SSK_Abstractions`**: Telemetry protocols (`AudioTelemetryPipelineProtocol`, `MusicUnderstandingProtocol`), domain models (`HierarchicalAudioFrame`, `MacroTelemetry`, `StemTelemetry`, `SignalTelemetry`). | |
| - **`SSK_Foundation`**: Diagnostics (`MXPerformanceTracer`) and cryptographic key management (`VaultKeyManager`). | |
| - **`SSK_Demucs_MLX`**: MLX neural model architecture (`DemucsModel`, `WeightLoader`). | |
| ### System Framework Dependencies | |
| - **`Accelerate` (`vDSP`)**: Vector math, FFT computation, and matrix operations. | |
| - **`AVFoundation`**: Audio reading, writing, and format conversion (`AVAudioPCMBuffer`, `AVAudioFile`). | |
| - **`CoreML`**: Fallback execution of compiled CoreML models. | |
| - **`CoreMedia`**: High-precision timestamps (`CMTime`, `CMTimeRange`). | |
| - **`CryptoKit`**: AES-GCM encryption for stored stem audio files. | |
| - **`MusicUnderstanding`**: System framework for high-level musical feature analysis. | |
| --- | |
| ## 5. Architectural Position | |
| ```text | |
| SSK_Abstractions | |
| ▲ | |
| │ | |
| SSK_Hardware | |
| ▲ ▲ | |
| ┌──────────────┘ └──────────────┐ | |
| │ │ | |
| SSK_Foundation SSK_Demucs_MLX | |
| ▲ | |
| │ | |
| mlx-swift | |
| ``` | |
Xet Storage Details
- Size:
- 5.44 kB
- Xet hash:
- db0b6d38393b53fc94f3cf54dfde9144a330c9ba04ad0d8210ed84ac5a583d80
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.