Spaces:
Running
Running
Upload folder using huggingface_hub
Browse files- .claude/handoff.md +94 -0
- .claude/vision-reference.md +111 -0
- MusicManager.js +268 -248
- game.js +57 -21
.claude/handoff.md
ADDED
|
@@ -0,0 +1,94 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Hyperspace Jam — Session Handoff (2026-03-23)
|
| 2 |
+
|
| 3 |
+
## STATUS: Working, needs per-finger sound refinement
|
| 4 |
+
|
| 5 |
+
Core app is stable: hand tracking, pad drone, sub-bass, drums, psychedelic visuals all work.
|
| 6 |
+
Per-finger sounds exist in code but need testing/tuning after the rampTo crash fix.
|
| 7 |
+
|
| 8 |
+
## Links
|
| 9 |
+
- **HF Space**: https://huggingface.co/spaces/Solshine/hyperspace-jam
|
| 10 |
+
- **Direct URL**: https://solshine-hyperspace-jam.static.hf.space/index.html
|
| 11 |
+
- **GitHub**: https://github.com/SolshineCode/hyperspace-jam-v2
|
| 12 |
+
|
| 13 |
+
## Deploy Command
|
| 14 |
+
```bash
|
| 15 |
+
cd /c/Users/caleb/hyperspace-jam-v2-clean
|
| 16 |
+
cp /c/Users/caleb/hyperspace-jam-v2/*.js /c/Users/caleb/hyperspace-jam-v2/*.html /c/Users/caleb/hyperspace-jam-v2/*.css .
|
| 17 |
+
python -c "from huggingface_hub import HfApi; HfApi().upload_folder(folder_path='.', repo_id='Solshine/hyperspace-jam', repo_type='space', ignore_patterns=['.git*'])"
|
| 18 |
+
```
|
| 19 |
+
|
| 20 |
+
## What Works
|
| 21 |
+
- Webcam + MediaPipe hand tracking (2 hands) in HF Spaces
|
| 22 |
+
- Pad drone with harmony voice (root + detuned 5th), smooth pitch glide
|
| 23 |
+
- Sub-bass layer with wobble LFO (hand spread = wobble speed)
|
| 24 |
+
- Hand height (Y) → pitch across 5 octaves (C1 sub-bass to F5)
|
| 25 |
+
- Thumb-index pinch → volume (normalized by palm size, works at any distance)
|
| 26 |
+
- Proximity to camera → lowpass filter sweep + reverb/delay wet
|
| 27 |
+
- Fist gesture → cycle pad presets (Hypnotic Sub / Acid Growl / Trance Wash)
|
| 28 |
+
- Spacebar PANIC → kills all sound + 1 second mute window
|
| 29 |
+
- Hand 2 drums via DrumManager (index=kick, middle=snare, ring=hihat, pinky=clap)
|
| 30 |
+
- Psychedelic hue-rotating webcam filter (fast, saturated)
|
| 31 |
+
- SVG turbulence displacement (organic per-pixel warping)
|
| 32 |
+
- Smoke/wave CSS distortion
|
| 33 |
+
- Poincaré hyperbolic shader background (audio-reactive, breathing)
|
| 34 |
+
- Mandala geometry (pentagram, rays, rings between fingers)
|
| 35 |
+
- Shape pinch detection + tessellation shader
|
| 36 |
+
- Per-finger labels on both hands
|
| 37 |
+
- Controls hint panel (frosted glass, bottom center)
|
| 38 |
+
- Pre-allocated geometry pools (ShapeManager + MandalaVisualizer) for stability
|
| 39 |
+
|
| 40 |
+
## PRIORITY 1: Per-Finger Sound Refinement
|
| 41 |
+
|
| 42 |
+
### The root cause we found
|
| 43 |
+
A `Tone.js rampTo()` with near-zero values threw `RangeError` EVERY FRAME, which crashed `_updateHands()`, which killed ALL tracking AND sound. Fixed by removing the offending rampTo and wrapping in try/catch.
|
| 44 |
+
|
| 45 |
+
### Current finger synth architecture (MusicManager.js)
|
| 46 |
+
**Hand 1 (synth hand) finger synths — all pre-allocated in start():**
|
| 47 |
+
- INDEX: MonoSynth "Acid Squelch" (sawtooth + filter sweep) → connected to delay
|
| 48 |
+
- MIDDLE: FMSynth "Laser Zap" (modulationIndex=25, square+sawtooth) → delay
|
| 49 |
+
- RING: MetalSynth "Crystal Chime" (resonance=3000, 1.5 octaves) → delay
|
| 50 |
+
- PINKY: MembraneSynth "Sub Drop" (pitchDecay=0.3, 6 octaves) → limiter
|
| 51 |
+
|
| 52 |
+
**Hand 2 (drum hand) finger synths — pre-allocated in start():**
|
| 53 |
+
- INDEX: MembraneSynth kick (8 octaves) → limiter
|
| 54 |
+
- MIDDLE: NoiseSynth "Riser" through AutoFilter → delay
|
| 55 |
+
- RING: NoiseSynth "Stutter" (rapid 4x bursts) → limiter
|
| 56 |
+
- PINKY: NoiseSynth "Crash" into dedicated Reverb(decay=8) → limiter
|
| 57 |
+
|
| 58 |
+
### What needs work next session
|
| 59 |
+
1. **Test with the crash fix** — the rampTo fix may have unblocked everything
|
| 60 |
+
2. **Verify finger extension values** — add console.log in updateGesture to see actual ext values
|
| 61 |
+
3. **The curl→extend threshold (0.25→0.35)** may need adjustment based on actual data
|
| 62 |
+
4. **Consider going back to the arpeggiator pattern** as the user liked — add finger sounds ON TOP of the arp, not replacing it
|
| 63 |
+
5. **The original arpeggiator's finger detection worked** because it used game.js's `_getFingerStates()` which is proven. Our custom extension calculation may be wrong.
|
| 64 |
+
|
| 65 |
+
### Key insight from user
|
| 66 |
+
"The arpeggiator worked so well though" — the working arpeggiator-based version had reliable hand detection because it didn't have per-frame errors crashing the update loop. With the crash fixed, the current version should track properly. But if finger detection is still unreliable, consider using `_getFingerStates()` (boolean up/down, proven to work) instead of continuous extension values.
|
| 67 |
+
|
| 68 |
+
## Key Files
|
| 69 |
+
| File | Purpose | Status |
|
| 70 |
+
|------|---------|--------|
|
| 71 |
+
| game.js | Main orchestrator (TRANSPILED ~1500 lines) | Many surgical edits, fragile |
|
| 72 |
+
| MusicManager.js | 10-finger psybass EDM engine | Crash fixed, needs testing |
|
| 73 |
+
| DrumManager.js | Drum sequencer | Stable, unchanged |
|
| 74 |
+
| WaveformVisualizer.js | Poincaré shader + breathing | Working |
|
| 75 |
+
| MandalaVisualizer.js | Sacred geometry | Pre-allocated pools, stable |
|
| 76 |
+
| ShapeManager.js | Pinch shapes + tessellation | Pre-allocated pools, stable |
|
| 77 |
+
| ShapeTessellationShader.js | GLSL for shape interiors | `ctr` fix applied |
|
| 78 |
+
| DisplacementFilter.js | SVG turbulence + smoke waves | Working |
|
| 79 |
+
| index.html | UI + controls hint | Updated labels |
|
| 80 |
+
| styles.css | Dark kiosk styling | Working |
|
| 81 |
+
|
| 82 |
+
## Effects Chain
|
| 83 |
+
```
|
| 84 |
+
Pad + Harmony → Filter → Chorus → Reverb → Limiter → Destination
|
| 85 |
+
Finger synths → Delay → Reverb → Limiter → Destination
|
| 86 |
+
Sub-bass → Limiter → Destination (direct, no effects)
|
| 87 |
+
Percussion → Limiter → Destination (direct, punchy)
|
| 88 |
+
Reverb → Analyser (for visualization)
|
| 89 |
+
```
|
| 90 |
+
|
| 91 |
+
## Remaining Phase 3 Features
|
| 92 |
+
- Phase 3.6: Multiplayer jam session (numHands: 4, treble+bass split)
|
| 93 |
+
- Dynamic shape internal tessellation (ShapeTessellationShader exists but untested)
|
| 94 |
+
- Attract mode logic (30s timer)
|
.claude/vision-reference.md
ADDED
|
@@ -0,0 +1,111 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Hyperspace Jam — Vision Reference: Dynamic Geometric Shapes with Internal Hyperbolic Tessellation
|
| 2 |
+
|
| 3 |
+
## Source
|
| 4 |
+
Instagram reel: https://www.instagram.com/reel/DTYN4uwCtuG/
|
| 5 |
+
Described by video analysis agent 2026-03-22.
|
| 6 |
+
|
| 7 |
+
## Core Interaction Model
|
| 8 |
+
|
| 9 |
+
### Dynamic Anchor Points
|
| 10 |
+
- **Pinch gesture** (thumb + index touching) = single anchor point
|
| 11 |
+
- **2 anchors** = line (two parallel lines connecting points)
|
| 12 |
+
- **3 anchors** = triangle
|
| 13 |
+
- **4 anchors** = quadrilateral
|
| 14 |
+
- System smoothly transitions between states as anchors are added/removed
|
| 15 |
+
|
| 16 |
+
### Anchor Sources
|
| 17 |
+
- Thumb tip (landmark 4)
|
| 18 |
+
- Index tip (landmark 8)
|
| 19 |
+
- Middle tip (landmark 12)
|
| 20 |
+
- Pinch = thumb+index merged into one point
|
| 21 |
+
- Both hands can contribute anchors simultaneously
|
| 22 |
+
|
| 23 |
+
### Shape Rendering (Layer 1: Wireframe)
|
| 24 |
+
- **Nodes**: Small hollow white squares with solid white dot center
|
| 25 |
+
- **Edges**: Crisp solid white lines connecting nodes
|
| 26 |
+
- **Z-depth**: Line thickness and node size scale inversely with distance from camera
|
| 27 |
+
- Close to camera = thick lines, large nodes
|
| 28 |
+
- Far from camera = thin lines, small nodes
|
| 29 |
+
- No physics/gravity — pure tension between tracked points
|
| 30 |
+
|
| 31 |
+
## Internal Visualization (Layer 2: Hyperbolic Geometry)
|
| 32 |
+
|
| 33 |
+
### When a closed shape forms (held > ~0.2s):
|
| 34 |
+
- Internal space fills with **Poincaré disk tessellation**
|
| 35 |
+
- Central focal point inside the shape
|
| 36 |
+
- Tessellating shapes get smaller and denser toward edges (hyperbolic compression)
|
| 37 |
+
- Pattern dynamically re-tessellates as shape morphs with finger movement
|
| 38 |
+
|
| 39 |
+
### Shape-specific tessellations:
|
| 40 |
+
- **Line**: 1D hyperbolic waves and interference rings between parallel lines
|
| 41 |
+
- **Triangle**: Triangle-based Poincaré tessellation with central singularity
|
| 42 |
+
- **Quadrilateral**: Quad-based tiling (e.g., {4,5} non-Euclidean tiling)
|
| 43 |
+
|
| 44 |
+
### Dynamic behavior:
|
| 45 |
+
- Pattern scales and re-tessellates in real-time as fingers move
|
| 46 |
+
- Stretching fingers = tessellation scales, colors intensify
|
| 47 |
+
- Collapsing shape = reverse singularity wink-out effect
|
| 48 |
+
|
| 49 |
+
## Psychedelic Coloring (Layer 3: DMT-Chrome)
|
| 50 |
+
|
| 51 |
+
### Color palette:
|
| 52 |
+
- Deep blues, electric purples, neon pinks, vibrant greens
|
| 53 |
+
- Iridescent shifting between colors (not static)
|
| 54 |
+
- Colors flow along curved non-Euclidean geometry toward focal point
|
| 55 |
+
|
| 56 |
+
### Effects:
|
| 57 |
+
- Biological pulsation rhythm (edges flare brighter at peaks)
|
| 58 |
+
- Fractal particles drift from nodes during transitions
|
| 59 |
+
- Maximum stretch = most intense iridescence
|
| 60 |
+
- Close to camera = overwhelming magenta/turquoise flares
|
| 61 |
+
|
| 62 |
+
## Audio Hooks
|
| 63 |
+
- Shape proximity to camera → distortion/low-pass filter
|
| 64 |
+
- Close = distorted/filtered, far = clear
|
| 65 |
+
- Pattern complexity peak synced with audio distortion peak
|
| 66 |
+
- Pulsation rhythm could sync with beat
|
| 67 |
+
|
| 68 |
+
## Multiplayer Jam Session (3+ hands detected)
|
| 69 |
+
|
| 70 |
+
### Core Concept
|
| 71 |
+
When MediaPipe detects more than 2 hands, the system enters **multiplayer jam mode**. Each player pair gets distinct musical roles that complement each other — not just duplicates.
|
| 72 |
+
|
| 73 |
+
### Player Role Assignment
|
| 74 |
+
- **Player 1 (hands 0-1)**: Treble / Lead — higher octave range (C3-C6), brighter synth presets, faster arpeggio patterns
|
| 75 |
+
- **Player 2 (hands 2-3)**: Bass / Foundation — lower octave range (C1-C3), deeper presets (Acid Bass, sub-sine), slower patterns, heavier reverb
|
| 76 |
+
- Could also split as: melodic vs rhythmic, or arp vs pad
|
| 77 |
+
|
| 78 |
+
### Musical Separation Strategy
|
| 79 |
+
- Each player pair routes through its own PolySynth instance with distinct preset
|
| 80 |
+
- Separate effects chains (Player 1: bright delay, Player 2: deep reverb)
|
| 81 |
+
- Octave offset so they naturally harmonize rather than clash
|
| 82 |
+
- Same scale (C Minor Pentatonic) but different registers = instant musical compatibility
|
| 83 |
+
- BPM is shared — both players lock to the same Transport clock
|
| 84 |
+
|
| 85 |
+
### Visual Differentiation
|
| 86 |
+
- Player 1 mandala/geometry: cyan + magenta palette
|
| 87 |
+
- Player 2 mandala/geometry: gold + green palette
|
| 88 |
+
- Internal Poincaré tessellations use different {P,Q} tilings per player
|
| 89 |
+
- Inter-player geometry: lines connecting across players' hands create larger mandala structures
|
| 90 |
+
|
| 91 |
+
### Detection Logic
|
| 92 |
+
- MediaPipe HandLandmarker already supports `numHands: 4`
|
| 93 |
+
- Hands 0-1 = Player 1, Hands 2-3 = Player 2
|
| 94 |
+
- If only 2 hands detected, single-player mode (current behavior)
|
| 95 |
+
- If 3-4 hands detected, split into two players automatically
|
| 96 |
+
- UI indicator shows "JAM SESSION" when multiplayer is active
|
| 97 |
+
|
| 98 |
+
### Scaling Considerations
|
| 99 |
+
- 4 hands = 84 landmarks at 30fps — still performant
|
| 100 |
+
- Two separate PolySynth instances may need careful volume balancing
|
| 101 |
+
- Master bus limiter (-2dB) protects speakers regardless of player count
|
| 102 |
+
|
| 103 |
+
## Implementation Priority for Hyperspace Jam
|
| 104 |
+
1. Pinch detection → merge thumb+index into single anchor
|
| 105 |
+
2. Dynamic shape state (line → triangle → quad) based on active anchors
|
| 106 |
+
3. Internal Poincaré tessellation shader filling the shape geometry
|
| 107 |
+
4. DMT-chrome iridescent color flow
|
| 108 |
+
5. Z-depth line thickness + node size scaling
|
| 109 |
+
6. Audio filter modulation from shape proximity
|
| 110 |
+
7. Pulsation and particle effects
|
| 111 |
+
8. **Multiplayer jam session** — detect 3-4 hands, split into treble+bass roles
|
MusicManager.js
CHANGED
|
@@ -1,20 +1,17 @@
|
|
| 1 |
import * as Tone from 'https://esm.sh/tone';
|
| 2 |
|
| 3 |
-
//
|
| 4 |
-
// Every finger
|
| 5 |
export class MusicManager {
|
| 6 |
constructor() {
|
| 7 |
-
|
| 8 |
-
this.
|
| 9 |
-
this.activePatterns = this.padSynths; // backward compat — game.js checks .activePatterns.has(i)
|
| 10 |
|
| 11 |
-
// Effects
|
| 12 |
this.reverb = null;
|
| 13 |
this.delay = null;
|
| 14 |
this.chorus = null;
|
| 15 |
this.analyser = null;
|
| 16 |
|
| 17 |
-
// State tracking
|
| 18 |
this.isStarted = false;
|
| 19 |
this.handVolumes = new Map();
|
| 20 |
this.fingerCooldowns = new Map();
|
|
@@ -24,191 +21,162 @@ export class MusicManager {
|
|
| 24 |
this.PERC_COOLDOWN_MS = 150;
|
| 25 |
this.HAND_VELOCITY_THRESHOLD = 0.15;
|
| 26 |
|
| 27 |
-
// Finger -> semitone interval mapping for synth hand melodic triggers
|
| 28 |
this.fingerIntervals = {
|
| 29 |
-
index: 0,
|
| 30 |
-
middle: 3, // minor third
|
| 31 |
-
ring: 7, // perfect fifth
|
| 32 |
-
pinky: 10 // minor seventh
|
| 33 |
};
|
| 34 |
|
| 35 |
-
// Extended C Minor Pentatonic
|
| 36 |
this.scale = [
|
| 37 |
-
'C1',
|
| 38 |
-
'
|
| 39 |
-
'C3', 'Eb3', 'F3', 'G3', 'Bb3',
|
| 40 |
-
'C4', 'Eb4', 'F4', 'G4', 'Bb4',
|
| 41 |
-
'C5', 'Eb5', 'F5'
|
| 42 |
];
|
| 43 |
|
| 44 |
-
// Pad timbre presets
|
| 45 |
this.padPresets = [
|
| 46 |
-
{
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
},
|
| 51 |
-
{
|
| 52 |
-
name: 'Acid Growl',
|
| 53 |
-
oscillator: { type: 'sawtooth' },
|
| 54 |
-
envelope: { attack: 0.1, decay: 0.3, sustain: 0.5, release: 0.4 }
|
| 55 |
-
},
|
| 56 |
-
{
|
| 57 |
-
name: 'Trance Wash',
|
| 58 |
-
oscillator: { type: 'triangle' },
|
| 59 |
-
envelope: { attack: 0.5, decay: 0.6, sustain: 0.55, release: 0.6 }
|
| 60 |
-
}
|
| 61 |
];
|
| 62 |
this.currentSynthIndex = 0;
|
| 63 |
|
| 64 |
-
// Previous extension tracking for both hands
|
| 65 |
this._prevExtensions = {};
|
| 66 |
-
this._prevDrumExtensions = { index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 67 |
-
this._drumFingerCooldowns = { index: 0, middle: 0, ring: 0, pinky: 0 };
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
}
|
| 69 |
|
| 70 |
async start() {
|
| 71 |
if (this.isStarted) return;
|
| 72 |
-
|
| 73 |
await Tone.start();
|
| 74 |
|
| 75 |
-
// === EFFECTS CHAIN ===
|
| 76 |
-
|
| 77 |
-
// Master limiter
|
| 78 |
this.limiter = new Tone.Limiter(-3).toDestination();
|
| 79 |
|
| 80 |
-
// Dark
|
| 81 |
-
this.reverb = new Tone.Reverb({
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
}).connect(this.limiter);
|
| 86 |
|
| 87 |
-
//
|
| 88 |
-
this.
|
| 89 |
-
delayTime: '8n.',
|
| 90 |
-
feedback: 0.35,
|
| 91 |
-
wet: 0.2
|
| 92 |
-
}).connect(this.reverb);
|
| 93 |
-
|
| 94 |
-
// Chorus for pad width (no Phaser — CPU savings)
|
| 95 |
-
this.chorus = new Tone.Chorus({
|
| 96 |
-
frequency: 2,
|
| 97 |
-
delayTime: 4,
|
| 98 |
-
depth: 0.7
|
| 99 |
-
}).connect(this.reverb);
|
| 100 |
this.chorus.start();
|
| 101 |
|
| 102 |
-
//
|
| 103 |
this.filter = new Tone.Filter(16000, 'lowpass').connect(this.chorus);
|
| 104 |
|
| 105 |
-
// Analyser for visualization
|
| 106 |
this.analyser = new Tone.Analyser('waveform', 1024);
|
| 107 |
this.reverb.connect(this.analyser);
|
| 108 |
|
| 109 |
-
//
|
| 110 |
|
|
|
|
| 111 |
this.fingerSynths = {};
|
| 112 |
-
|
| 113 |
-
// INDEX: "Acid Squelch" — MonoSynth with sawtooth + filter sweep
|
| 114 |
this.fingerSynths.index = new Tone.MonoSynth({
|
| 115 |
oscillator: { type: 'sawtooth' },
|
| 116 |
-
filter: {
|
| 117 |
-
|
| 118 |
-
type: 'lowpass',
|
| 119 |
-
rolloff: -24
|
| 120 |
-
},
|
| 121 |
-
envelope: { attack: 0.005, decay: 0.2, sustain: 0.1, release: 0.3 },
|
| 122 |
filterEnvelope: {
|
| 123 |
-
attack: 0.001,
|
| 124 |
-
|
| 125 |
-
sustain: 0.05,
|
| 126 |
-
release: 0.2,
|
| 127 |
-
baseFrequency: 200,
|
| 128 |
-
octaves: 4,
|
| 129 |
-
exponent: 2
|
| 130 |
}
|
| 131 |
}).connect(this.delay);
|
| 132 |
this.fingerSynths.index.volume.value = -4;
|
| 133 |
|
| 134 |
-
// MIDDLE:
|
| 135 |
this.fingerSynths.middle = new Tone.FMSynth({
|
| 136 |
-
harmonicity:
|
| 137 |
-
modulationIndex:
|
| 138 |
oscillator: { type: 'square' },
|
| 139 |
-
envelope: { attack: 0.
|
| 140 |
modulation: { type: 'sawtooth' },
|
| 141 |
-
modulationEnvelope: { attack: 0.
|
| 142 |
}).connect(this.delay);
|
| 143 |
this.fingerSynths.middle.volume.value = -6;
|
| 144 |
|
| 145 |
-
// RING:
|
| 146 |
this.fingerSynths.ring = new Tone.MetalSynth({
|
| 147 |
-
harmonicity: 12,
|
| 148 |
-
|
| 149 |
-
resonance: 3000,
|
| 150 |
-
octaves: 1.5,
|
| 151 |
envelope: { attack: 0.001, decay: 0.4, sustain: 0, release: 0.3 }
|
| 152 |
}).connect(this.delay);
|
| 153 |
this.fingerSynths.ring.volume.value = -8;
|
| 154 |
|
| 155 |
-
// PINKY:
|
|
|
|
| 156 |
this.fingerSynths.pinky = new Tone.MembraneSynth({
|
| 157 |
-
pitchDecay: 0.3,
|
| 158 |
-
octaves: 6,
|
| 159 |
oscillator: { type: 'sine' },
|
| 160 |
envelope: { attack: 0.001, decay: 0.8, sustain: 0, release: 0.5 }
|
| 161 |
-
}).connect(this.
|
| 162 |
this.fingerSynths.pinky.volume.value = -2;
|
| 163 |
|
| 164 |
-
// ===
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 165 |
|
|
|
|
| 166 |
this.drumFingerSynths = {};
|
| 167 |
|
| 168 |
-
//
|
| 169 |
this.drumFingerSynths.index = new Tone.MembraneSynth({
|
| 170 |
-
pitchDecay: 0.08,
|
| 171 |
-
octaves: 8,
|
| 172 |
oscillator: { type: 'sine' },
|
| 173 |
envelope: { attack: 0.001, decay: 0.5, sustain: 0, release: 0.5 }
|
| 174 |
-
}).connect(this.limiter);
|
| 175 |
this.drumFingerSynths.index.volume.value = -4;
|
| 176 |
|
| 177 |
-
//
|
| 178 |
this.drumFingerSynths.middle = new Tone.NoiseSynth({
|
| 179 |
noise: { type: 'pink' },
|
| 180 |
envelope: { attack: 0.05, decay: 0.6, sustain: 0, release: 0.3 }
|
| 181 |
});
|
| 182 |
this._riserFilter = new Tone.AutoFilter({
|
| 183 |
-
frequency: 4,
|
| 184 |
-
baseFrequency: 200,
|
| 185 |
-
octaves: 6,
|
| 186 |
filter: { type: 'bandpass', Q: 2 }
|
| 187 |
}).connect(this.delay);
|
| 188 |
this._riserFilter.start();
|
| 189 |
this.drumFingerSynths.middle.connect(this._riserFilter);
|
| 190 |
this.drumFingerSynths.middle.volume.value = -8;
|
| 191 |
|
| 192 |
-
//
|
| 193 |
this.drumFingerSynths.ring = new Tone.NoiseSynth({
|
| 194 |
noise: { type: 'white' },
|
| 195 |
envelope: { attack: 0.001, decay: 0.02, sustain: 0, release: 0.01 }
|
| 196 |
-
}).connect(this.limiter);
|
| 197 |
this.drumFingerSynths.ring.volume.value = -10;
|
| 198 |
|
| 199 |
-
//
|
| 200 |
-
this._crashReverb = new Tone.Reverb({
|
| 201 |
-
decay: 8,
|
| 202 |
-
preDelay: 0.01,
|
| 203 |
-
wet: 0.9
|
| 204 |
-
}).connect(this.limiter);
|
| 205 |
this.drumFingerSynths.pinky = new Tone.NoiseSynth({
|
| 206 |
noise: { type: 'white' },
|
| 207 |
envelope: { attack: 0.001, decay: 0.15, sustain: 0, release: 0.1 }
|
| 208 |
}).connect(this._crashReverb);
|
| 209 |
this.drumFingerSynths.pinky.volume.value = -6;
|
| 210 |
|
| 211 |
-
//
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 212 |
this.kickSynth = this.drumFingerSynths.index;
|
| 213 |
this.hatSynth = new Tone.NoiseSynth({
|
| 214 |
noise: { type: 'white' },
|
|
@@ -217,33 +185,36 @@ export class MusicManager {
|
|
| 217 |
this.hatSynth.volume.value = -12;
|
| 218 |
this.pluckSynth = { releaseAll: () => {} };
|
| 219 |
|
| 220 |
-
// === SUB-BASS
|
| 221 |
this.subBass = new Tone.Synth({
|
| 222 |
oscillator: { type: 'sine' },
|
| 223 |
envelope: { attack: 0.3, decay: 0, sustain: 1, release: 0.8 }
|
| 224 |
-
}).connect(this.limiter);
|
| 225 |
this.subBass.volume.value = -18;
|
| 226 |
|
| 227 |
-
|
| 228 |
-
this.wobbleLFO = new Tone.LFO({
|
| 229 |
-
frequency: 0.3,
|
| 230 |
-
min: -22,
|
| 231 |
-
max: -12
|
| 232 |
-
});
|
| 233 |
this.wobbleLFO.connect(this.subBass.volume);
|
| 234 |
this.wobbleLFO.start();
|
| 235 |
|
|
|
|
|
|
|
|
|
|
| 236 |
this.isStarted = true;
|
| 237 |
-
console.log('Psychedelic
|
| 238 |
}
|
| 239 |
|
| 240 |
-
// ---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 241 |
|
| 242 |
startArpeggio(handId, rootNote) {
|
| 243 |
if (!this.isStarted || this.padSynths.has(handId) || this._panicMuted) return;
|
| 244 |
|
| 245 |
const preset = this.padPresets[this.currentSynthIndex];
|
| 246 |
-
|
| 247 |
const pad = new Tone.Synth({
|
| 248 |
oscillator: { ...preset.oscillator },
|
| 249 |
envelope: { ...preset.envelope }
|
|
@@ -262,13 +233,19 @@ export class MusicManager {
|
|
| 262 |
pad.triggerAttack(freq, Tone.now());
|
| 263 |
harmonyPad.triggerAttack(freq * 1.498, Tone.now());
|
| 264 |
|
| 265 |
-
if (this.subBass)
|
| 266 |
-
this.subBass.triggerAttack(freq / 2, Tone.now());
|
| 267 |
-
}
|
| 268 |
|
| 269 |
this.padSynths.set(handId, { synth: pad, harmonySynth: harmonyPad, currentRoot: rootNote });
|
| 270 |
this.handVolumes.set(handId, 0.2);
|
| 271 |
-
this.fingerCooldowns.set(handId, { index: 0, middle: 0, ring: 0, pinky: 0 });
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 272 |
}
|
| 273 |
|
| 274 |
updateArpeggio(handId, newRootNote) {
|
|
@@ -277,12 +254,14 @@ export class MusicManager {
|
|
| 277 |
|
| 278 |
const freq = Tone.Frequency(newRootNote).toFrequency();
|
| 279 |
padData.synth.frequency.rampTo(freq, 0.15);
|
| 280 |
-
if (padData.harmonySynth)
|
| 281 |
-
|
| 282 |
-
|
| 283 |
-
|
| 284 |
-
|
| 285 |
-
|
|
|
|
|
|
|
| 286 |
padData.currentRoot = newRootNote;
|
| 287 |
}
|
| 288 |
|
|
@@ -294,9 +273,7 @@ export class MusicManager {
|
|
| 294 |
this.handVolumes.set(handId, clamped);
|
| 295 |
const db = -30 + clamped * 26;
|
| 296 |
padData.synth.volume.rampTo(db, 0.1);
|
| 297 |
-
if (padData.harmonySynth)
|
| 298 |
-
padData.harmonySynth.volume.rampTo(db - 6, 0.1);
|
| 299 |
-
}
|
| 300 |
}
|
| 301 |
|
| 302 |
stopArpeggio(handId) {
|
|
@@ -308,13 +285,19 @@ export class MusicManager {
|
|
| 308 |
this.subBass.triggerRelease(Tone.now());
|
| 309 |
}
|
| 310 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 311 |
if (!this._pendingDisposals) this._pendingDisposals = new Set();
|
| 312 |
const synths = [padData.synth, padData.harmonySynth].filter(Boolean);
|
| 313 |
for (const synth of synths) {
|
| 314 |
if (!this._pendingDisposals.has(synth)) {
|
| 315 |
this._pendingDisposals.add(synth);
|
| 316 |
setTimeout(() => {
|
| 317 |
-
try { synth.dispose(); } catch(e) {
|
| 318 |
this._pendingDisposals.delete(synth);
|
| 319 |
}, 2000);
|
| 320 |
}
|
|
@@ -325,41 +308,45 @@ export class MusicManager {
|
|
| 325 |
}
|
| 326 |
}
|
| 327 |
|
| 328 |
-
// ---
|
| 329 |
-
//
|
| 330 |
-
//
|
| 331 |
|
| 332 |
updateGesture(handId, gestureData) {
|
| 333 |
if (!this.isStarted || this._panicMuted) return;
|
| 334 |
|
| 335 |
const { fingerStates, handVelocity, rootNote } = gestureData;
|
| 336 |
-
|
| 337 |
const now = performance.now();
|
| 338 |
const cooldowns = this.fingerCooldowns.get(handId);
|
| 339 |
if (!cooldowns) return;
|
| 340 |
|
| 341 |
if (!this._prevExtensions[handId]) {
|
| 342 |
-
this._prevExtensions[handId] = { index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 343 |
}
|
| 344 |
const prevExt = this._prevExtensions[handId];
|
| 345 |
|
| 346 |
-
|
| 347 |
-
|
| 348 |
-
|
| 349 |
-
ring: fingerStates.ring ? 0.8 : 0.1,
|
| 350 |
-
pinky: fingerStates.pinky ? 0.8 : 0.1
|
| 351 |
};
|
| 352 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 353 |
const rootFreq = Tone.Frequency(rootNote).toFrequency();
|
| 354 |
|
| 355 |
-
// ===
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 356 |
if (this.fingerSynths.index) {
|
| 357 |
-
|
| 358 |
-
const cutoff = 200 + ext.index * 4000;
|
| 359 |
-
this.fingerSynths.index.filter.frequency.value = cutoff;
|
| 360 |
-
} catch(e) {}
|
| 361 |
-
// Trigger on curl→extend transition
|
| 362 |
-
if (prevExt.index < 0.2 && ext.index > 0.35 && (now - cooldowns.index) > this.FINGER_COOLDOWN_MS) {
|
| 363 |
cooldowns.index = now;
|
| 364 |
this.fingerSynths.index.triggerAttackRelease(
|
| 365 |
Tone.Frequency(rootFreq).toNote(), '4n', Tone.now(), 0.8
|
|
@@ -367,57 +354,99 @@ export class MusicManager {
|
|
| 367 |
}
|
| 368 |
}
|
| 369 |
|
| 370 |
-
// ===
|
|
|
|
|
|
|
|
|
|
| 371 |
if (this.fingerSynths.middle) {
|
| 372 |
-
|
| 373 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 374 |
cooldowns.middle = now;
|
| 375 |
-
const zapFreq = rootFreq * Math.pow(2, 3/12);
|
| 376 |
this.fingerSynths.middle.triggerAttackRelease(
|
| 377 |
Tone.Frequency(zapFreq).toNote(), '8n', Tone.now(), 0.9
|
| 378 |
);
|
| 379 |
}
|
| 380 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 381 |
|
| 382 |
-
// ===
|
|
|
|
|
|
|
|
|
|
| 383 |
if (this.fingerSynths.ring) {
|
| 384 |
-
|
| 385 |
-
if (prevExt.ring < 0.2 && ext.ring > 0.35 && (now - cooldowns.ring) > this.FINGER_COOLDOWN_MS) {
|
| 386 |
cooldowns.ring = now;
|
| 387 |
-
const chimeFreq = rootFreq * Math.pow(2, 7/12);
|
| 388 |
this.fingerSynths.ring.triggerAttackRelease(chimeFreq, '4n', Tone.now(), 0.8);
|
| 389 |
}
|
| 390 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 391 |
|
| 392 |
-
// ===
|
|
|
|
|
|
|
|
|
|
| 393 |
if (this.fingerSynths.pinky) {
|
| 394 |
-
|
| 395 |
-
if (prevExt.pinky < 0.2 && ext.pinky > 0.35 && (now - cooldowns.pinky) > this.FINGER_COOLDOWN_MS) {
|
| 396 |
cooldowns.pinky = now;
|
| 397 |
this.fingerSynths.pinky.triggerAttackRelease('C1', '4n', Tone.now(), 0.9);
|
| 398 |
}
|
| 399 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 400 |
|
| 401 |
-
// ===
|
| 402 |
-
|
| 403 |
-
|
| 404 |
-
|
| 405 |
-
|
| 406 |
-
|
| 407 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 408 |
|
| 409 |
// Store for next frame
|
| 410 |
-
prevExt.
|
| 411 |
-
prevExt.
|
| 412 |
-
prevExt.
|
| 413 |
-
prevExt.
|
|
|
|
| 414 |
|
| 415 |
// Percussive hits on sharp hand movement
|
| 416 |
if ((now - this.percCooldown) > this.PERC_COOLDOWN_MS) {
|
| 417 |
const vx = handVelocity?.x || 0;
|
| 418 |
const vy = handVelocity?.y || 0;
|
| 419 |
const magnitude = Math.sqrt(vx * vx + vy * vy);
|
| 420 |
-
|
| 421 |
if (magnitude > this.HAND_VELOCITY_THRESHOLD) {
|
| 422 |
this.percCooldown = now;
|
| 423 |
if (vy > this.HAND_VELOCITY_THRESHOLD) {
|
|
@@ -429,8 +458,7 @@ export class MusicManager {
|
|
| 429 |
}
|
| 430 |
}
|
| 431 |
|
| 432 |
-
// ---
|
| 433 |
-
// HYBRID: triggers PLUS continuous modulation — every finger always does something
|
| 434 |
|
| 435 |
updateDrumGesture(gestureData) {
|
| 436 |
if (!this.isStarted || this._panicMuted) return;
|
|
@@ -439,96 +467,100 @@ export class MusicManager {
|
|
| 439 |
const prevExt = this._prevDrumExtensions;
|
| 440 |
const cooldowns = this._drumFingerCooldowns;
|
| 441 |
|
| 442 |
-
const
|
| 443 |
-
|
| 444 |
-
index: fingerStates.index ? 0.8 : 0.1,
|
| 445 |
-
middle: fingerStates.middle ? 0.8 : 0.1,
|
| 446 |
-
ring: fingerStates.ring ? 0.8 : 0.1,
|
| 447 |
-
pinky: fingerStates.pinky ? 0.8 : 0.1
|
| 448 |
};
|
| 449 |
|
| 450 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 451 |
if (this.drumFingerSynths.index) {
|
| 452 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 453 |
cooldowns.index = now;
|
| 454 |
this.drumFingerSynths.index.triggerAttackRelease('C1', '8n', Tone.now(), 0.9);
|
| 455 |
}
|
| 456 |
}
|
| 457 |
|
| 458 |
-
//
|
| 459 |
if (this._riserFilter) {
|
| 460 |
-
this._riserFilter.baseFrequency = 100 +
|
|
|
|
| 461 |
}
|
| 462 |
if (this.drumFingerSynths.middle) {
|
| 463 |
-
if (prevExt.middle < 0.2 &&
|
| 464 |
cooldowns.middle = now;
|
| 465 |
this.drumFingerSynths.middle.triggerAttackRelease('4n', Tone.now(), 0.7);
|
| 466 |
}
|
| 467 |
}
|
| 468 |
|
| 469 |
-
//
|
| 470 |
if (this.drumFingerSynths.ring) {
|
| 471 |
-
if (prevExt.ring < 0.2 &&
|
| 472 |
cooldowns.ring = now;
|
| 473 |
-
|
|
|
|
|
|
|
| 474 |
setTimeout(() => {
|
| 475 |
try { this.drumFingerSynths.ring.triggerAttackRelease('64n', Tone.now(), 0.6); } catch(e) {}
|
| 476 |
-
},
|
| 477 |
}
|
| 478 |
}
|
| 479 |
}
|
| 480 |
|
| 481 |
-
//
|
| 482 |
if (this._crashReverb) {
|
| 483 |
-
this._crashReverb.wet.value = 0.
|
| 484 |
}
|
| 485 |
if (this.drumFingerSynths.pinky) {
|
| 486 |
-
if (prevExt.pinky < 0.2 &&
|
| 487 |
cooldowns.pinky = now;
|
| 488 |
this.drumFingerSynths.pinky.triggerAttackRelease('8n', Tone.now(), 0.8);
|
| 489 |
}
|
| 490 |
}
|
| 491 |
|
| 492 |
-
//
|
| 493 |
-
const
|
| 494 |
if (this.chorus) {
|
| 495 |
-
this.chorus.depth = 0.3 +
|
| 496 |
}
|
| 497 |
|
| 498 |
-
prevExt.
|
| 499 |
-
prevExt.
|
| 500 |
-
prevExt.
|
| 501 |
-
prevExt.
|
|
|
|
| 502 |
}
|
| 503 |
|
| 504 |
-
// --- Finger Expression (
|
| 505 |
|
| 506 |
updateFingerExpression(params) {
|
| 507 |
if (!this.isStarted) return;
|
| 508 |
|
| 509 |
-
const { middleFinger, ringFinger, pinkyFinger, handSpread } = params;
|
| 510 |
-
|
| 511 |
-
// Ring finger -> delay feedback
|
| 512 |
-
if (this.delay) {
|
| 513 |
-
this.delay.feedback.value = 0.15 + ringFinger * 0.45;
|
| 514 |
-
}
|
| 515 |
-
|
| 516 |
-
// Pinky -> reverb depth
|
| 517 |
-
if (this.reverb) {
|
| 518 |
-
this.reverb.wet.value = 0.2 + pinkyFinger * 0.4;
|
| 519 |
-
}
|
| 520 |
|
| 521 |
-
// Hand spread
|
| 522 |
if (this.wobbleLFO) {
|
| 523 |
this.wobbleLFO.frequency.value = 0.2 + handSpread * 6;
|
| 524 |
}
|
| 525 |
-
|
| 526 |
-
// Hand spread -> pad detune
|
| 527 |
this.padSynths.forEach(padData => {
|
| 528 |
padData.synth.detune.rampTo(handSpread * 40, 0.1);
|
| 529 |
-
if (padData.harmonySynth)
|
| 530 |
-
padData.harmonySynth.detune.rampTo(-handSpread * 25, 0.1);
|
| 531 |
-
}
|
| 532 |
});
|
| 533 |
}
|
| 534 |
|
|
@@ -564,23 +596,17 @@ export class MusicManager {
|
|
| 564 |
const cutoff = 16000 * Math.pow(0.075, value);
|
| 565 |
this.filter.frequency.rampTo(cutoff, 0.2);
|
| 566 |
}
|
| 567 |
-
if (this.reverb)
|
| 568 |
-
|
| 569 |
-
|
| 570 |
-
if (this.delay) {
|
| 571 |
-
this.delay.wet.value = 0.15 + value * 0.2;
|
| 572 |
-
}
|
| 573 |
-
if (this.chorus) {
|
| 574 |
-
this.chorus.depth = 0.4 + value * 0.4;
|
| 575 |
-
}
|
| 576 |
}
|
| 577 |
|
| 578 |
-
// --- PANIC
|
|
|
|
| 579 |
panic() {
|
| 580 |
this._panicMuted = true;
|
| 581 |
setTimeout(() => { this._panicMuted = false; }, 1000);
|
| 582 |
|
| 583 |
-
// Stop all pads
|
| 584 |
this.padSynths.forEach((padData) => {
|
| 585 |
try { padData.synth.triggerRelease(Tone.now()); } catch(e) {}
|
| 586 |
try { if (padData.harmonySynth) padData.harmonySynth.triggerRelease(Tone.now()); } catch(e) {}
|
|
@@ -593,44 +619,38 @@ export class MusicManager {
|
|
| 593 |
this.handVolumes.clear();
|
| 594 |
this.fingerCooldowns.clear();
|
| 595 |
|
| 596 |
-
|
| 597 |
-
|
| 598 |
-
|
| 599 |
-
}
|
|
|
|
| 600 |
|
| 601 |
-
// Silence all synth hand finger synths
|
| 602 |
if (this.fingerSynths) {
|
| 603 |
for (const finger of ['index', 'middle', 'ring', 'pinky']) {
|
| 604 |
try { this.fingerSynths[finger].triggerRelease(Tone.now()); } catch(e) {}
|
| 605 |
}
|
| 606 |
}
|
| 607 |
-
|
| 608 |
-
// Silence all drum hand finger synths
|
| 609 |
if (this.drumFingerSynths) {
|
| 610 |
for (const finger of ['index', 'middle', 'ring', 'pinky']) {
|
| 611 |
try { this.drumFingerSynths[finger].triggerRelease(Tone.now()); } catch(e) {}
|
| 612 |
}
|
| 613 |
}
|
| 614 |
|
| 615 |
-
// Kill wobble LFO temporarily
|
| 616 |
if (this.wobbleLFO) {
|
| 617 |
this.wobbleLFO.stop();
|
| 618 |
setTimeout(() => { try { this.wobbleLFO.start(); } catch(e) {} }, 1000);
|
| 619 |
}
|
| 620 |
|
| 621 |
-
|
| 622 |
-
if (this.filter) {
|
| 623 |
-
this.filter.frequency.value = 16000;
|
| 624 |
-
}
|
| 625 |
-
|
| 626 |
-
// Reset effect wet levels
|
| 627 |
if (this.reverb) this.reverb.wet.value = 0.35;
|
| 628 |
if (this.delay) this.delay.wet.value = 0.2;
|
| 629 |
if (this.chorus) this.chorus.depth = 0.6;
|
|
|
|
| 630 |
|
| 631 |
-
|
| 632 |
-
this.
|
| 633 |
-
this.
|
|
|
|
| 634 |
|
| 635 |
console.log('PANIC — all sound killed, muted for 1 second');
|
| 636 |
}
|
|
|
|
| 1 |
import * as Tone from 'https://esm.sh/tone';
|
| 2 |
|
| 3 |
+
// Psychedelic EDM Synth Engine — Comprehensive Hand-Distance Control
|
| 4 |
+
// Every finger's distance from palm continuously drives dramatic synth parameters
|
| 5 |
export class MusicManager {
|
| 6 |
constructor() {
|
| 7 |
+
this.padSynths = new Map();
|
| 8 |
+
this.activePatterns = this.padSynths;
|
|
|
|
| 9 |
|
|
|
|
| 10 |
this.reverb = null;
|
| 11 |
this.delay = null;
|
| 12 |
this.chorus = null;
|
| 13 |
this.analyser = null;
|
| 14 |
|
|
|
|
| 15 |
this.isStarted = false;
|
| 16 |
this.handVolumes = new Map();
|
| 17 |
this.fingerCooldowns = new Map();
|
|
|
|
| 21 |
this.PERC_COOLDOWN_MS = 150;
|
| 22 |
this.HAND_VELOCITY_THRESHOLD = 0.15;
|
| 23 |
|
|
|
|
| 24 |
this.fingerIntervals = {
|
| 25 |
+
index: 0, middle: 3, ring: 7, pinky: 10
|
|
|
|
|
|
|
|
|
|
| 26 |
};
|
| 27 |
|
| 28 |
+
// Extended C Minor Pentatonic
|
| 29 |
this.scale = [
|
| 30 |
+
'C1','Eb1','G1','Bb1','C2','Eb2','F2','G2','Bb2',
|
| 31 |
+
'C3','Eb3','F3','G3','Bb3','C4','Eb4','F4','G4','Bb4','C5','Eb5','F5'
|
|
|
|
|
|
|
|
|
|
| 32 |
];
|
| 33 |
|
|
|
|
| 34 |
this.padPresets = [
|
| 35 |
+
{ name: 'Hypnotic Sub', oscillator: { type: 'sine' }, envelope: { attack: 0.4, decay: 0.5, sustain: 0.6, release: 0.5 } },
|
| 36 |
+
{ name: 'Acid Growl', oscillator: { type: 'sawtooth' }, envelope: { attack: 0.1, decay: 0.3, sustain: 0.5, release: 0.4 } },
|
| 37 |
+
{ name: 'Trance Wash', oscillator: { type: 'triangle' }, envelope: { attack: 0.5, decay: 0.6, sustain: 0.55, release: 0.6 } },
|
| 38 |
+
{ name: 'PWM Swirl', oscillator: { type: 'pwm', modulationFrequency: 0.5 }, envelope: { attack: 0.3, decay: 0.4, sustain: 0.7, release: 0.6 } },
|
| 39 |
+
{ name: 'Fat Square', oscillator: { type: 'fatsquare', spread: 30 }, envelope: { attack: 0.2, decay: 0.3, sustain: 0.6, release: 0.5 } }
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
];
|
| 41 |
this.currentSynthIndex = 0;
|
| 42 |
|
|
|
|
| 43 |
this._prevExtensions = {};
|
| 44 |
+
this._prevDrumExtensions = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 45 |
+
this._drumFingerCooldowns = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 46 |
+
|
| 47 |
+
// Smoothed finger distances for continuous control (prevents jitter)
|
| 48 |
+
this._smoothDist = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 49 |
+
this._smoothDrumDist = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 50 |
+
this._SMOOTH_ALPHA = 0.25; // smoothing factor
|
| 51 |
}
|
| 52 |
|
| 53 |
async start() {
|
| 54 |
if (this.isStarted) return;
|
|
|
|
| 55 |
await Tone.start();
|
| 56 |
|
| 57 |
+
// === MASTER EFFECTS CHAIN ===
|
|
|
|
|
|
|
| 58 |
this.limiter = new Tone.Limiter(-3).toDestination();
|
| 59 |
|
| 60 |
+
// Dark reverb
|
| 61 |
+
this.reverb = new Tone.Reverb({ decay: 6, preDelay: 0.03, wet: 0.35 }).connect(this.limiter);
|
| 62 |
+
|
| 63 |
+
// Ping-pong delay
|
| 64 |
+
this.delay = new Tone.PingPongDelay({ delayTime: '8n.', feedback: 0.35, wet: 0.2 }).connect(this.reverb);
|
|
|
|
| 65 |
|
| 66 |
+
// Chorus
|
| 67 |
+
this.chorus = new Tone.Chorus({ frequency: 2, delayTime: 4, depth: 0.7 }).connect(this.reverb);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
this.chorus.start();
|
| 69 |
|
| 70 |
+
// Master proximity filter
|
| 71 |
this.filter = new Tone.Filter(16000, 'lowpass').connect(this.chorus);
|
| 72 |
|
| 73 |
+
// Analyser for visualization
|
| 74 |
this.analyser = new Tone.Analyser('waveform', 1024);
|
| 75 |
this.reverb.connect(this.analyser);
|
| 76 |
|
| 77 |
+
// NOTE: Thumb + Index are volume control (pinch) — no continuous effects on them
|
| 78 |
|
| 79 |
+
// === INDEX: Acid Squelch — continuously pitched, filter sweeps with distance ===
|
| 80 |
this.fingerSynths = {};
|
|
|
|
|
|
|
| 81 |
this.fingerSynths.index = new Tone.MonoSynth({
|
| 82 |
oscillator: { type: 'sawtooth' },
|
| 83 |
+
filter: { Q: 8, type: 'lowpass', rolloff: -24 },
|
| 84 |
+
envelope: { attack: 0.005, decay: 0.2, sustain: 0.3, release: 0.3 },
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
filterEnvelope: {
|
| 86 |
+
attack: 0.001, decay: 0.15, sustain: 0.1, release: 0.2,
|
| 87 |
+
baseFrequency: 200, octaves: 4, exponent: 2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
}
|
| 89 |
}).connect(this.delay);
|
| 90 |
this.fingerSynths.index.volume.value = -4;
|
| 91 |
|
| 92 |
+
// === MIDDLE: FM Chaos — distance controls modulation index (clean → insane) ===
|
| 93 |
this.fingerSynths.middle = new Tone.FMSynth({
|
| 94 |
+
harmonicity: 3,
|
| 95 |
+
modulationIndex: 1,
|
| 96 |
oscillator: { type: 'square' },
|
| 97 |
+
envelope: { attack: 0.01, decay: 0.3, sustain: 0.2, release: 0.3 },
|
| 98 |
modulation: { type: 'sawtooth' },
|
| 99 |
+
modulationEnvelope: { attack: 0.01, decay: 0.15, sustain: 0.1, release: 0.1 }
|
| 100 |
}).connect(this.delay);
|
| 101 |
this.fingerSynths.middle.volume.value = -6;
|
| 102 |
|
| 103 |
+
// === RING: Shimmer Delay — distance controls delay time + feedback (tight → infinite) ===
|
| 104 |
this.fingerSynths.ring = new Tone.MetalSynth({
|
| 105 |
+
harmonicity: 12, modulationIndex: 24,
|
| 106 |
+
resonance: 3000, octaves: 1.5,
|
|
|
|
|
|
|
| 107 |
envelope: { attack: 0.001, decay: 0.4, sustain: 0, release: 0.3 }
|
| 108 |
}).connect(this.delay);
|
| 109 |
this.fingerSynths.ring.volume.value = -8;
|
| 110 |
|
| 111 |
+
// === PINKY: Sub Drop + Bitcrusher — distance controls crush depth ===
|
| 112 |
+
this.bitcrusher = new Tone.BitCrusher({ bits: 16 }).connect(this.limiter);
|
| 113 |
this.fingerSynths.pinky = new Tone.MembraneSynth({
|
| 114 |
+
pitchDecay: 0.3, octaves: 6,
|
|
|
|
| 115 |
oscillator: { type: 'sine' },
|
| 116 |
envelope: { attack: 0.001, decay: 0.8, sustain: 0, release: 0.5 }
|
| 117 |
+
}).connect(this.bitcrusher);
|
| 118 |
this.fingerSynths.pinky.volume.value = -2;
|
| 119 |
|
| 120 |
+
// === CONTINUOUS SYNTHS (always sounding when hand present) ===
|
| 121 |
+
// Only on middle/ring/pinky — NOT thumb/index (those are volume fingers)
|
| 122 |
+
|
| 123 |
+
// Middle continuous: FM pad that gets chaotic with distance
|
| 124 |
+
this.middleContinuous = new Tone.FMSynth({
|
| 125 |
+
harmonicity: 2, modulationIndex: 0.5,
|
| 126 |
+
oscillator: { type: 'sine' },
|
| 127 |
+
envelope: { attack: 0.4, decay: 0, sustain: 1, release: 0.6 },
|
| 128 |
+
modulation: { type: 'triangle' },
|
| 129 |
+
modulationEnvelope: { attack: 0.4, decay: 0, sustain: 1, release: 0.6 }
|
| 130 |
+
}).connect(this.chorus);
|
| 131 |
+
this.middleContinuous.volume.value = -20;
|
| 132 |
|
| 133 |
+
// === DRUM HAND SYNTHS (hand 1) ===
|
| 134 |
this.drumFingerSynths = {};
|
| 135 |
|
| 136 |
+
// Index: Kick
|
| 137 |
this.drumFingerSynths.index = new Tone.MembraneSynth({
|
| 138 |
+
pitchDecay: 0.08, octaves: 8,
|
|
|
|
| 139 |
oscillator: { type: 'sine' },
|
| 140 |
envelope: { attack: 0.001, decay: 0.5, sustain: 0, release: 0.5 }
|
| 141 |
+
}).connect(this.limiter);
|
| 142 |
this.drumFingerSynths.index.volume.value = -4;
|
| 143 |
|
| 144 |
+
// Middle: Sci-Fi Riser
|
| 145 |
this.drumFingerSynths.middle = new Tone.NoiseSynth({
|
| 146 |
noise: { type: 'pink' },
|
| 147 |
envelope: { attack: 0.05, decay: 0.6, sustain: 0, release: 0.3 }
|
| 148 |
});
|
| 149 |
this._riserFilter = new Tone.AutoFilter({
|
| 150 |
+
frequency: 4, baseFrequency: 200, octaves: 6,
|
|
|
|
|
|
|
| 151 |
filter: { type: 'bandpass', Q: 2 }
|
| 152 |
}).connect(this.delay);
|
| 153 |
this._riserFilter.start();
|
| 154 |
this.drumFingerSynths.middle.connect(this._riserFilter);
|
| 155 |
this.drumFingerSynths.middle.volume.value = -8;
|
| 156 |
|
| 157 |
+
// Ring: Granular Stutter
|
| 158 |
this.drumFingerSynths.ring = new Tone.NoiseSynth({
|
| 159 |
noise: { type: 'white' },
|
| 160 |
envelope: { attack: 0.001, decay: 0.02, sustain: 0, release: 0.01 }
|
| 161 |
+
}).connect(this.limiter);
|
| 162 |
this.drumFingerSynths.ring.volume.value = -10;
|
| 163 |
|
| 164 |
+
// Pinky: Reverb Crash
|
| 165 |
+
this._crashReverb = new Tone.Reverb({ decay: 8, preDelay: 0.01, wet: 0.9 }).connect(this.limiter);
|
|
|
|
|
|
|
|
|
|
|
|
|
| 166 |
this.drumFingerSynths.pinky = new Tone.NoiseSynth({
|
| 167 |
noise: { type: 'white' },
|
| 168 |
envelope: { attack: 0.001, decay: 0.15, sustain: 0, release: 0.1 }
|
| 169 |
}).connect(this._crashReverb);
|
| 170 |
this.drumFingerSynths.pinky.volume.value = -6;
|
| 171 |
|
| 172 |
+
// Thumb: Drum hand thumb = tempo wobble synth
|
| 173 |
+
this.drumThumbSynth = new Tone.Synth({
|
| 174 |
+
oscillator: { type: 'sine' },
|
| 175 |
+
envelope: { attack: 0.01, decay: 0.3, sustain: 0, release: 0.2 }
|
| 176 |
+
}).connect(this.limiter);
|
| 177 |
+
this.drumThumbSynth.volume.value = -10;
|
| 178 |
+
|
| 179 |
+
// Legacy references
|
| 180 |
this.kickSynth = this.drumFingerSynths.index;
|
| 181 |
this.hatSynth = new Tone.NoiseSynth({
|
| 182 |
noise: { type: 'white' },
|
|
|
|
| 185 |
this.hatSynth.volume.value = -12;
|
| 186 |
this.pluckSynth = { releaseAll: () => {} };
|
| 187 |
|
| 188 |
+
// === SUB-BASS ===
|
| 189 |
this.subBass = new Tone.Synth({
|
| 190 |
oscillator: { type: 'sine' },
|
| 191 |
envelope: { attack: 0.3, decay: 0, sustain: 1, release: 0.8 }
|
| 192 |
+
}).connect(this.limiter);
|
| 193 |
this.subBass.volume.value = -18;
|
| 194 |
|
| 195 |
+
this.wobbleLFO = new Tone.LFO({ frequency: 0.3, min: -22, max: -12 });
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 196 |
this.wobbleLFO.connect(this.subBass.volume);
|
| 197 |
this.wobbleLFO.start();
|
| 198 |
|
| 199 |
+
// Track continuous synth state
|
| 200 |
+
this._continuousActive = false;
|
| 201 |
+
|
| 202 |
this.isStarted = true;
|
| 203 |
+
console.log('Psychedelic EDM engine v2 ready — 10-finger distance-from-palm control');
|
| 204 |
}
|
| 205 |
|
| 206 |
+
// --- Smoothing helper ---
|
| 207 |
+
_smooth(target, key, raw) {
|
| 208 |
+
target[key] += (raw - target[key]) * this._SMOOTH_ALPHA;
|
| 209 |
+
return target[key];
|
| 210 |
+
}
|
| 211 |
+
|
| 212 |
+
// --- Pad Management ---
|
| 213 |
|
| 214 |
startArpeggio(handId, rootNote) {
|
| 215 |
if (!this.isStarted || this.padSynths.has(handId) || this._panicMuted) return;
|
| 216 |
|
| 217 |
const preset = this.padPresets[this.currentSynthIndex];
|
|
|
|
| 218 |
const pad = new Tone.Synth({
|
| 219 |
oscillator: { ...preset.oscillator },
|
| 220 |
envelope: { ...preset.envelope }
|
|
|
|
| 233 |
pad.triggerAttack(freq, Tone.now());
|
| 234 |
harmonyPad.triggerAttack(freq * 1.498, Tone.now());
|
| 235 |
|
| 236 |
+
if (this.subBass) this.subBass.triggerAttack(freq / 2, Tone.now());
|
|
|
|
|
|
|
| 237 |
|
| 238 |
this.padSynths.set(handId, { synth: pad, harmonySynth: harmonyPad, currentRoot: rootNote });
|
| 239 |
this.handVolumes.set(handId, 0.2);
|
| 240 |
+
this.fingerCooldowns.set(handId, { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 });
|
| 241 |
+
|
| 242 |
+
// Start continuous synths (middle only — thumb/index are volume fingers)
|
| 243 |
+
if (!this._continuousActive) {
|
| 244 |
+
this._continuousActive = true;
|
| 245 |
+
try {
|
| 246 |
+
this.middleContinuous.triggerAttack(freq * 1.498, Tone.now());
|
| 247 |
+
} catch(e) {}
|
| 248 |
+
}
|
| 249 |
}
|
| 250 |
|
| 251 |
updateArpeggio(handId, newRootNote) {
|
|
|
|
| 254 |
|
| 255 |
const freq = Tone.Frequency(newRootNote).toFrequency();
|
| 256 |
padData.synth.frequency.rampTo(freq, 0.15);
|
| 257 |
+
if (padData.harmonySynth) padData.harmonySynth.frequency.rampTo(freq * 1.498, 0.15);
|
| 258 |
+
if (this.subBass) this.subBass.frequency.rampTo(freq / 2, 0.2);
|
| 259 |
+
|
| 260 |
+
// Update continuous synth to track root
|
| 261 |
+
try {
|
| 262 |
+
if (this.middleContinuous) this.middleContinuous.frequency.rampTo(freq * 1.498, 0.2);
|
| 263 |
+
} catch(e) {}
|
| 264 |
+
|
| 265 |
padData.currentRoot = newRootNote;
|
| 266 |
}
|
| 267 |
|
|
|
|
| 273 |
this.handVolumes.set(handId, clamped);
|
| 274 |
const db = -30 + clamped * 26;
|
| 275 |
padData.synth.volume.rampTo(db, 0.1);
|
| 276 |
+
if (padData.harmonySynth) padData.harmonySynth.volume.rampTo(db - 6, 0.1);
|
|
|
|
|
|
|
| 277 |
}
|
| 278 |
|
| 279 |
stopArpeggio(handId) {
|
|
|
|
| 285 |
this.subBass.triggerRelease(Tone.now());
|
| 286 |
}
|
| 287 |
|
| 288 |
+
// Release continuous synths
|
| 289 |
+
if (this.padSynths.size <= 1) {
|
| 290 |
+
this._continuousActive = false;
|
| 291 |
+
try { this.middleContinuous.triggerRelease(Tone.now()); } catch(e) {}
|
| 292 |
+
}
|
| 293 |
+
|
| 294 |
if (!this._pendingDisposals) this._pendingDisposals = new Set();
|
| 295 |
const synths = [padData.synth, padData.harmonySynth].filter(Boolean);
|
| 296 |
for (const synth of synths) {
|
| 297 |
if (!this._pendingDisposals.has(synth)) {
|
| 298 |
this._pendingDisposals.add(synth);
|
| 299 |
setTimeout(() => {
|
| 300 |
+
try { synth.dispose(); } catch(e) {}
|
| 301 |
this._pendingDisposals.delete(synth);
|
| 302 |
}, 2000);
|
| 303 |
}
|
|
|
|
| 308 |
}
|
| 309 |
}
|
| 310 |
|
| 311 |
+
// --- SYNTH HAND (hand 0): Comprehensive Distance-from-Palm Control ---
|
| 312 |
+
// Each finger's distance ratio (0=curled to palm, 1=fully extended) CONTINUOUSLY
|
| 313 |
+
// drives dramatic parameter changes. Triggers still fire on transitions.
|
| 314 |
|
| 315 |
updateGesture(handId, gestureData) {
|
| 316 |
if (!this.isStarted || this._panicMuted) return;
|
| 317 |
|
| 318 |
const { fingerStates, handVelocity, rootNote } = gestureData;
|
|
|
|
| 319 |
const now = performance.now();
|
| 320 |
const cooldowns = this.fingerCooldowns.get(handId);
|
| 321 |
if (!cooldowns) return;
|
| 322 |
|
| 323 |
if (!this._prevExtensions[handId]) {
|
| 324 |
+
this._prevExtensions[handId] = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 325 |
}
|
| 326 |
const prevExt = this._prevExtensions[handId];
|
| 327 |
|
| 328 |
+
// Use fingerDistances (Euclidean from palm) as primary control, fall back to extensions
|
| 329 |
+
const dist = gestureData.fingerDistances || gestureData.fingerExtensions || {
|
| 330 |
+
thumb: 0.1, index: 0.1, middle: 0.1, ring: 0.1, pinky: 0.1
|
|
|
|
|
|
|
| 331 |
};
|
| 332 |
|
| 333 |
+
// Smooth all distances to prevent jitter
|
| 334 |
+
const d = {};
|
| 335 |
+
for (const f of ['thumb', 'index', 'middle', 'ring', 'pinky']) {
|
| 336 |
+
d[f] = this._smooth(this._smoothDist, f, dist[f] || 0);
|
| 337 |
+
}
|
| 338 |
+
|
| 339 |
const rootFreq = Tone.Frequency(rootNote).toFrequency();
|
| 340 |
|
| 341 |
+
// ====================================================================
|
| 342 |
+
// THUMB + INDEX: These are the VOLUME CONTROL fingers (pinch distance)
|
| 343 |
+
// Do NOT assign continuous effects here — they'd be forced on whenever
|
| 344 |
+
// volume is up. Only trigger one-shots on curl→extend transitions.
|
| 345 |
+
// ====================================================================
|
| 346 |
+
|
| 347 |
+
// INDEX: Acid Squelch — trigger only (no continuous filter tied to distance)
|
| 348 |
if (this.fingerSynths.index) {
|
| 349 |
+
if (prevExt.index < 0.2 && d.index > 0.35 && (now - cooldowns.index) > this.FINGER_COOLDOWN_MS) {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 350 |
cooldowns.index = now;
|
| 351 |
this.fingerSynths.index.triggerAttackRelease(
|
| 352 |
Tone.Frequency(rootFreq).toNote(), '4n', Tone.now(), 0.8
|
|
|
|
| 354 |
}
|
| 355 |
}
|
| 356 |
|
| 357 |
+
// ====================================================================
|
| 358 |
+
// MIDDLE: FM Chaos — Distance = modulation index (0.5→40)
|
| 359 |
+
// Close = clean tone, extended = insane FM screeching
|
| 360 |
+
// ====================================================================
|
| 361 |
if (this.fingerSynths.middle) {
|
| 362 |
+
try {
|
| 363 |
+
this.fingerSynths.middle.modulationIndex.value = 0.5 + Math.pow(d.middle, 2) * 40;
|
| 364 |
+
this.fingerSynths.middle.harmonicity.value = 2 + d.middle * 10;
|
| 365 |
+
} catch(e) {}
|
| 366 |
+
|
| 367 |
+
if (prevExt.middle < 0.2 && d.middle > 0.35 && (now - cooldowns.middle) > this.FINGER_COOLDOWN_MS) {
|
| 368 |
cooldowns.middle = now;
|
| 369 |
+
const zapFreq = rootFreq * Math.pow(2, 3/12);
|
| 370 |
this.fingerSynths.middle.triggerAttackRelease(
|
| 371 |
Tone.Frequency(zapFreq).toNote(), '8n', Tone.now(), 0.9
|
| 372 |
);
|
| 373 |
}
|
| 374 |
}
|
| 375 |
+
// Continuous FM pad
|
| 376 |
+
if (this.middleContinuous) {
|
| 377 |
+
try {
|
| 378 |
+
this.middleContinuous.modulationIndex.value = 0.2 + Math.pow(d.middle, 2) * 15;
|
| 379 |
+
this.middleContinuous.volume.rampTo(-28 + d.middle * 14, 0.05);
|
| 380 |
+
} catch(e) {}
|
| 381 |
+
}
|
| 382 |
|
| 383 |
+
// ====================================================================
|
| 384 |
+
// RING: Shimmer/Delay — Distance = delay feedback + time warping
|
| 385 |
+
// Close = dry/tight, extended = infinite echoing shimmer
|
| 386 |
+
// ====================================================================
|
| 387 |
if (this.fingerSynths.ring) {
|
| 388 |
+
if (prevExt.ring < 0.2 && d.ring > 0.35 && (now - cooldowns.ring) > this.FINGER_COOLDOWN_MS) {
|
|
|
|
| 389 |
cooldowns.ring = now;
|
| 390 |
+
const chimeFreq = rootFreq * Math.pow(2, 7/12);
|
| 391 |
this.fingerSynths.ring.triggerAttackRelease(chimeFreq, '4n', Tone.now(), 0.8);
|
| 392 |
}
|
| 393 |
}
|
| 394 |
+
if (this.delay) {
|
| 395 |
+
this.delay.feedback.value = 0.1 + d.ring * 0.65;
|
| 396 |
+
this.delay.wet.value = 0.05 + d.ring * 0.5;
|
| 397 |
+
// Modulate delay time subtly for chorus-like shimmer
|
| 398 |
+
try {
|
| 399 |
+
const delayMod = 0.15 + d.ring * 0.15; // 8th note dotted range
|
| 400 |
+
this.delay.delayTime.value = delayMod;
|
| 401 |
+
} catch(e) {}
|
| 402 |
+
}
|
| 403 |
|
| 404 |
+
// ====================================================================
|
| 405 |
+
// PINKY: Sub Drop + Bitcrusher — Distance = crush bits + reverb intensity
|
| 406 |
+
// Close = clean sub, extended = destroyed/crushed + massive reverb
|
| 407 |
+
// ====================================================================
|
| 408 |
if (this.fingerSynths.pinky) {
|
| 409 |
+
if (prevExt.pinky < 0.2 && d.pinky > 0.35 && (now - cooldowns.pinky) > this.FINGER_COOLDOWN_MS) {
|
|
|
|
| 410 |
cooldowns.pinky = now;
|
| 411 |
this.fingerSynths.pinky.triggerAttackRelease('C1', '4n', Tone.now(), 0.9);
|
| 412 |
}
|
| 413 |
}
|
| 414 |
+
if (this.bitcrusher) {
|
| 415 |
+
// 16 bits (clean) → 2 bits (destroyed)
|
| 416 |
+
this.bitcrusher.bits.value = Math.max(2, Math.round(16 - d.pinky * 14));
|
| 417 |
+
}
|
| 418 |
+
if (this.reverb) {
|
| 419 |
+
this.reverb.wet.value = 0.15 + d.pinky * 0.55;
|
| 420 |
+
}
|
| 421 |
|
| 422 |
+
// ====================================================================
|
| 423 |
+
// COMBINED: Hand spread → chorus + detuning + wobble speed
|
| 424 |
+
// ====================================================================
|
| 425 |
+
const spread = gestureData.handSpread || 0;
|
| 426 |
+
if (this.chorus) {
|
| 427 |
+
this.chorus.depth = 0.3 + spread * 0.7;
|
| 428 |
+
this.chorus.frequency.value = 1 + spread * 4;
|
| 429 |
+
}
|
| 430 |
+
if (this.wobbleLFO) {
|
| 431 |
+
this.wobbleLFO.frequency.value = 0.2 + spread * 8;
|
| 432 |
+
}
|
| 433 |
+
this.padSynths.forEach(padData => {
|
| 434 |
+
padData.synth.detune.rampTo(spread * 50, 0.1);
|
| 435 |
+
if (padData.harmonySynth) padData.harmonySynth.detune.rampTo(-spread * 30, 0.1);
|
| 436 |
+
});
|
| 437 |
|
| 438 |
// Store for next frame
|
| 439 |
+
prevExt.thumb = d.thumb;
|
| 440 |
+
prevExt.index = d.index;
|
| 441 |
+
prevExt.middle = d.middle;
|
| 442 |
+
prevExt.ring = d.ring;
|
| 443 |
+
prevExt.pinky = d.pinky;
|
| 444 |
|
| 445 |
// Percussive hits on sharp hand movement
|
| 446 |
if ((now - this.percCooldown) > this.PERC_COOLDOWN_MS) {
|
| 447 |
const vx = handVelocity?.x || 0;
|
| 448 |
const vy = handVelocity?.y || 0;
|
| 449 |
const magnitude = Math.sqrt(vx * vx + vy * vy);
|
|
|
|
| 450 |
if (magnitude > this.HAND_VELOCITY_THRESHOLD) {
|
| 451 |
this.percCooldown = now;
|
| 452 |
if (vy > this.HAND_VELOCITY_THRESHOLD) {
|
|
|
|
| 458 |
}
|
| 459 |
}
|
| 460 |
|
| 461 |
+
// --- DRUM HAND (hand 1): Distance-from-Palm Control ---
|
|
|
|
| 462 |
|
| 463 |
updateDrumGesture(gestureData) {
|
| 464 |
if (!this.isStarted || this._panicMuted) return;
|
|
|
|
| 467 |
const prevExt = this._prevDrumExtensions;
|
| 468 |
const cooldowns = this._drumFingerCooldowns;
|
| 469 |
|
| 470 |
+
const dist = gestureData.fingerDistances || gestureData.fingerExtensions || {
|
| 471 |
+
thumb: 0.1, index: 0.1, middle: 0.1, ring: 0.1, pinky: 0.1
|
|
|
|
|
|
|
|
|
|
|
|
|
| 472 |
};
|
| 473 |
|
| 474 |
+
const d = {};
|
| 475 |
+
for (const f of ['thumb', 'index', 'middle', 'ring', 'pinky']) {
|
| 476 |
+
d[f] = this._smooth(this._smoothDrumDist, f, dist[f] || 0);
|
| 477 |
+
}
|
| 478 |
+
|
| 479 |
+
// THUMB: Tempo wobble — distance triggers rhythmic tom hits at varying pitch
|
| 480 |
+
if (this.drumThumbSynth) {
|
| 481 |
+
if (prevExt.thumb < 0.2 && d.thumb > 0.35 && (now - (cooldowns.thumb || 0)) > 200) {
|
| 482 |
+
cooldowns.thumb = now;
|
| 483 |
+
const pitch = 60 + d.thumb * 200; // Hz
|
| 484 |
+
this.drumThumbSynth.triggerAttackRelease(pitch, '16n', Tone.now(), 0.7);
|
| 485 |
+
}
|
| 486 |
+
}
|
| 487 |
+
|
| 488 |
+
// INDEX: Kick — distance controls pitch decay depth
|
| 489 |
if (this.drumFingerSynths.index) {
|
| 490 |
+
try {
|
| 491 |
+
this.drumFingerSynths.index.pitchDecay = 0.02 + d.index * 0.2;
|
| 492 |
+
this.drumFingerSynths.index.octaves = 4 + d.index * 6;
|
| 493 |
+
} catch(e) {}
|
| 494 |
+
if (prevExt.index < 0.2 && d.index > 0.35 && (now - cooldowns.index) > this.FINGER_COOLDOWN_MS) {
|
| 495 |
cooldowns.index = now;
|
| 496 |
this.drumFingerSynths.index.triggerAttackRelease('C1', '8n', Tone.now(), 0.9);
|
| 497 |
}
|
| 498 |
}
|
| 499 |
|
| 500 |
+
// MIDDLE: Riser — distance = filter sweep position
|
| 501 |
if (this._riserFilter) {
|
| 502 |
+
this._riserFilter.baseFrequency = 100 + d.middle * 4000;
|
| 503 |
+
this._riserFilter.octaves = 2 + d.middle * 6;
|
| 504 |
}
|
| 505 |
if (this.drumFingerSynths.middle) {
|
| 506 |
+
if (prevExt.middle < 0.2 && d.middle > 0.35 && (now - cooldowns.middle) > 200) {
|
| 507 |
cooldowns.middle = now;
|
| 508 |
this.drumFingerSynths.middle.triggerAttackRelease('4n', Tone.now(), 0.7);
|
| 509 |
}
|
| 510 |
}
|
| 511 |
|
| 512 |
+
// RING: Stutter — distance controls burst count and speed
|
| 513 |
if (this.drumFingerSynths.ring) {
|
| 514 |
+
if (prevExt.ring < 0.2 && d.ring > 0.35 && (now - cooldowns.ring) > 250) {
|
| 515 |
cooldowns.ring = now;
|
| 516 |
+
const burstCount = Math.round(2 + d.ring * 8); // 2→10 hits
|
| 517 |
+
const burstSpeed = Math.max(15, 60 - d.ring * 40); // ms between hits
|
| 518 |
+
for (let j = 0; j < burstCount; j++) {
|
| 519 |
setTimeout(() => {
|
| 520 |
try { this.drumFingerSynths.ring.triggerAttackRelease('64n', Tone.now(), 0.6); } catch(e) {}
|
| 521 |
+
}, j * burstSpeed);
|
| 522 |
}
|
| 523 |
}
|
| 524 |
}
|
| 525 |
|
| 526 |
+
// PINKY: Reverb Crash — distance = reverb size + wet
|
| 527 |
if (this._crashReverb) {
|
| 528 |
+
this._crashReverb.wet.value = 0.3 + d.pinky * 0.7;
|
| 529 |
}
|
| 530 |
if (this.drumFingerSynths.pinky) {
|
| 531 |
+
if (prevExt.pinky < 0.2 && d.pinky > 0.35 && (now - cooldowns.pinky) > 300) {
|
| 532 |
cooldowns.pinky = now;
|
| 533 |
this.drumFingerSynths.pinky.triggerAttackRelease('8n', Tone.now(), 0.8);
|
| 534 |
}
|
| 535 |
}
|
| 536 |
|
| 537 |
+
// CONTINUOUS: chorus from average drum distance
|
| 538 |
+
const avgDist = (d.index + d.middle + d.ring + d.pinky) / 4;
|
| 539 |
if (this.chorus) {
|
| 540 |
+
this.chorus.depth = 0.3 + avgDist * 0.7;
|
| 541 |
}
|
| 542 |
|
| 543 |
+
prevExt.thumb = d.thumb;
|
| 544 |
+
prevExt.index = d.index;
|
| 545 |
+
prevExt.middle = d.middle;
|
| 546 |
+
prevExt.ring = d.ring;
|
| 547 |
+
prevExt.pinky = d.pinky;
|
| 548 |
}
|
| 549 |
|
| 550 |
+
// --- Finger Expression (legacy + enhanced) ---
|
| 551 |
|
| 552 |
updateFingerExpression(params) {
|
| 553 |
if (!this.isStarted) return;
|
| 554 |
|
| 555 |
+
const { fingerDistances, middleFinger, ringFinger, pinkyFinger, handSpread } = params;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 556 |
|
| 557 |
+
// Hand spread → wobble LFO speed + pad detune
|
| 558 |
if (this.wobbleLFO) {
|
| 559 |
this.wobbleLFO.frequency.value = 0.2 + handSpread * 6;
|
| 560 |
}
|
|
|
|
|
|
|
| 561 |
this.padSynths.forEach(padData => {
|
| 562 |
padData.synth.detune.rampTo(handSpread * 40, 0.1);
|
| 563 |
+
if (padData.harmonySynth) padData.harmonySynth.detune.rampTo(-handSpread * 25, 0.1);
|
|
|
|
|
|
|
| 564 |
});
|
| 565 |
}
|
| 566 |
|
|
|
|
| 596 |
const cutoff = 16000 * Math.pow(0.075, value);
|
| 597 |
this.filter.frequency.rampTo(cutoff, 0.2);
|
| 598 |
}
|
| 599 |
+
if (this.reverb) this.reverb.wet.value = 0.3 + value * 0.3;
|
| 600 |
+
if (this.delay) this.delay.wet.value = 0.15 + value * 0.2;
|
| 601 |
+
if (this.chorus) this.chorus.depth = 0.4 + value * 0.4;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 602 |
}
|
| 603 |
|
| 604 |
+
// --- PANIC ---
|
| 605 |
+
|
| 606 |
panic() {
|
| 607 |
this._panicMuted = true;
|
| 608 |
setTimeout(() => { this._panicMuted = false; }, 1000);
|
| 609 |
|
|
|
|
| 610 |
this.padSynths.forEach((padData) => {
|
| 611 |
try { padData.synth.triggerRelease(Tone.now()); } catch(e) {}
|
| 612 |
try { if (padData.harmonySynth) padData.harmonySynth.triggerRelease(Tone.now()); } catch(e) {}
|
|
|
|
| 619 |
this.handVolumes.clear();
|
| 620 |
this.fingerCooldowns.clear();
|
| 621 |
|
| 622 |
+
if (this.subBass) try { this.subBass.triggerRelease(Tone.now()); } catch(e) {}
|
| 623 |
+
|
| 624 |
+
// Release continuous synths
|
| 625 |
+
try { this.middleContinuous.triggerRelease(Tone.now()); } catch(e) {}
|
| 626 |
+
this._continuousActive = false;
|
| 627 |
|
|
|
|
| 628 |
if (this.fingerSynths) {
|
| 629 |
for (const finger of ['index', 'middle', 'ring', 'pinky']) {
|
| 630 |
try { this.fingerSynths[finger].triggerRelease(Tone.now()); } catch(e) {}
|
| 631 |
}
|
| 632 |
}
|
|
|
|
|
|
|
| 633 |
if (this.drumFingerSynths) {
|
| 634 |
for (const finger of ['index', 'middle', 'ring', 'pinky']) {
|
| 635 |
try { this.drumFingerSynths[finger].triggerRelease(Tone.now()); } catch(e) {}
|
| 636 |
}
|
| 637 |
}
|
| 638 |
|
|
|
|
| 639 |
if (this.wobbleLFO) {
|
| 640 |
this.wobbleLFO.stop();
|
| 641 |
setTimeout(() => { try { this.wobbleLFO.start(); } catch(e) {} }, 1000);
|
| 642 |
}
|
| 643 |
|
| 644 |
+
if (this.filter) this.filter.frequency.value = 16000;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 645 |
if (this.reverb) this.reverb.wet.value = 0.35;
|
| 646 |
if (this.delay) this.delay.wet.value = 0.2;
|
| 647 |
if (this.chorus) this.chorus.depth = 0.6;
|
| 648 |
+
if (this.bitcrusher) this.bitcrusher.bits.value = 16;
|
| 649 |
|
| 650 |
+
this._prevDrumExtensions = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 651 |
+
this._drumFingerCooldowns = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 652 |
+
this._smoothDist = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 653 |
+
this._smoothDrumDist = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 654 |
|
| 655 |
console.log('PANIC — all sound killed, muted for 1 second');
|
| 656 |
}
|
game.js
CHANGED
|
@@ -831,20 +831,40 @@ export var Game = /*#__PURE__*/ function() {
|
|
| 831 |
_this1.musicManager.updateArpeggio(i, note);
|
| 832 |
}
|
| 833 |
_this1.musicManager.updateArpeggioVolume(i, velocity);
|
| 834 |
-
// --- Finger
|
| 835 |
-
// Each finger's
|
| 836 |
var wrist = smoothedLandmarks[0];
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 837 |
var middleTip = smoothedLandmarks[12];
|
| 838 |
var middlePip = smoothedLandmarks[10];
|
| 839 |
var ringTip = smoothedLandmarks[16];
|
| 840 |
var ringPip = smoothedLandmarks[14];
|
| 841 |
var pinkyTip = smoothedLandmarks[20];
|
| 842 |
var pinkyPip = smoothedLandmarks[18];
|
| 843 |
-
// Finger extension: how far tip is above its PIP joint (normalized)
|
| 844 |
-
var palmSize = Math.abs(wrist.y - smoothedLandmarks[9].y) || 0.1;
|
| 845 |
var middleExt = Math.max(0, Math.min(1, (middlePip.y - middleTip.y) / palmSize));
|
| 846 |
var ringExt = Math.max(0, Math.min(1, (ringPip.y - ringTip.y) / palmSize));
|
| 847 |
var pinkyExt = Math.max(0, Math.min(1, (pinkyPip.y - pinkyTip.y) / palmSize));
|
|
|
|
| 848 |
// Hand spread: average distance between adjacent fingertips / palm size
|
| 849 |
var tips = [smoothedLandmarks[4], smoothedLandmarks[8], middleTip, ringTip, pinkyTip];
|
| 850 |
var spreadSum = 0;
|
|
@@ -855,14 +875,15 @@ export var Game = /*#__PURE__*/ function() {
|
|
| 855 |
}
|
| 856 |
var handSpread = Math.max(0, Math.min(1, (spreadSum / 4) / (palmSize * 2)));
|
| 857 |
_this1.musicManager.updateFingerExpression({
|
|
|
|
| 858 |
middleFinger: middleExt,
|
| 859 |
ringFinger: ringExt,
|
| 860 |
pinkyFinger: pinkyExt,
|
| 861 |
handSpread: handSpread
|
| 862 |
});
|
| 863 |
// --- Gesture-Driven Music ---
|
| 864 |
-
var indexExt = Math.max(0, Math.min(1, (smoothedLandmarks[6].y - smoothedLandmarks[8].y) / palmSize));
|
| 865 |
var fingerStates = {
|
|
|
|
| 866 |
index: indexExt > 0.3,
|
| 867 |
middle: middleExt > 0.3,
|
| 868 |
ring: ringExt > 0.3,
|
|
@@ -871,31 +892,32 @@ export var Game = /*#__PURE__*/ function() {
|
|
| 871 |
// Track previous states and compute velocities
|
| 872 |
if (!_this1._prevFingerStates) _this1._prevFingerStates = {};
|
| 873 |
if (!_this1._prevHandPos) _this1._prevHandPos = {};
|
| 874 |
-
|
|
|
|
| 875 |
var prevHP = _this1._prevHandPos[i] || { x: hand.anchorPos.x, y: hand.anchorPos.y };
|
|
|
|
| 876 |
var handVelX = hand.anchorPos.x - prevHP.x;
|
| 877 |
var handVelY = hand.anchorPos.y - prevHP.y;
|
| 878 |
_this1.musicManager.updateGesture(i, {
|
| 879 |
fingerStates: fingerStates,
|
| 880 |
prevFingerStates: prevFS,
|
|
|
|
|
|
|
| 881 |
fingerExtensions: {
|
|
|
|
| 882 |
index: indexExt,
|
| 883 |
middle: middleExt,
|
| 884 |
ring: ringExt,
|
| 885 |
pinky: pinkyExt
|
| 886 |
},
|
| 887 |
-
fingerVelocities: {
|
| 888 |
-
index: Math.abs(indexExt - (prevFS.index ? 1 : 0)),
|
| 889 |
-
middle: Math.abs(middleExt - (prevFS.middle ? 1 : 0)),
|
| 890 |
-
ring: Math.abs(ringExt - (prevFS.ring ? 1 : 0)),
|
| 891 |
-
pinky: Math.abs(pinkyExt - (prevFS.pinky ? 1 : 0))
|
| 892 |
-
},
|
| 893 |
handVelocity: { x: handVelX, y: handVelY },
|
|
|
|
| 894 |
rootNote: note,
|
| 895 |
volume: velocity
|
| 896 |
});
|
| 897 |
_this1._prevFingerStates[i] = fingerStates;
|
| 898 |
_this1._prevHandPos[i] = { x: hand.anchorPos.x, y: hand.anchorPos.y };
|
|
|
|
| 899 |
} else {
|
| 900 |
// If it is a fist, make sure the arpeggio is stopped
|
| 901 |
_this1.musicManager.stopArpeggio(i);
|
|
@@ -908,19 +930,33 @@ export var Game = /*#__PURE__*/ function() {
|
|
| 908 |
});
|
| 909 |
// Per-finger drum sounds via updateDrumGesture
|
| 910 |
var drumWrist = smoothedLandmarks[0];
|
| 911 |
-
var
|
| 912 |
-
var
|
| 913 |
-
var
|
| 914 |
-
|
| 915 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 916 |
if (_this1.musicManager.updateDrumGesture) {
|
| 917 |
_this1.musicManager.updateDrumGesture({
|
| 918 |
fingerStates: fingerStates,
|
|
|
|
| 919 |
fingerExtensions: {
|
| 920 |
-
|
| 921 |
-
|
| 922 |
-
|
| 923 |
-
|
|
|
|
| 924 |
}
|
| 925 |
});
|
| 926 |
}
|
|
|
|
| 831 |
_this1.musicManager.updateArpeggio(i, note);
|
| 832 |
}
|
| 833 |
_this1.musicManager.updateArpeggioVolume(i, velocity);
|
| 834 |
+
// --- Finger Distance-from-Palm Controls ---
|
| 835 |
+
// Each finger's Euclidean distance from palm center, normalized by palm size
|
| 836 |
var wrist = smoothedLandmarks[0];
|
| 837 |
+
var palmCenter = smoothedLandmarks[9]; // middle finger MCP = palm center
|
| 838 |
+
var palmDist = Math.sqrt(Math.pow(wrist.x - palmCenter.x, 2) + Math.pow(wrist.y - palmCenter.y, 2)) || 0.05;
|
| 839 |
+
// Fingertip landmarks: thumb=4, index=8, middle=12, ring=16, pinky=20
|
| 840 |
+
var fingerTips = {
|
| 841 |
+
thumb: smoothedLandmarks[4],
|
| 842 |
+
index: smoothedLandmarks[8],
|
| 843 |
+
middle: smoothedLandmarks[12],
|
| 844 |
+
ring: smoothedLandmarks[16],
|
| 845 |
+
pinky: smoothedLandmarks[20]
|
| 846 |
+
};
|
| 847 |
+
var fingerDistances = {};
|
| 848 |
+
for (var fname in fingerTips) {
|
| 849 |
+
var tip = fingerTips[fname];
|
| 850 |
+
var fdx = tip.x - palmCenter.x;
|
| 851 |
+
var fdy = tip.y - palmCenter.y;
|
| 852 |
+
var fdist = Math.sqrt(fdx * fdx + fdy * fdy);
|
| 853 |
+
// Normalize: 0 = touching palm, 1 = fully extended (~ 2x palm size)
|
| 854 |
+
fingerDistances[fname] = Math.max(0, Math.min(1, fdist / (palmDist * 2)));
|
| 855 |
+
}
|
| 856 |
+
// Also keep legacy extension values for compatibility
|
| 857 |
+
var palmSize = palmDist;
|
| 858 |
var middleTip = smoothedLandmarks[12];
|
| 859 |
var middlePip = smoothedLandmarks[10];
|
| 860 |
var ringTip = smoothedLandmarks[16];
|
| 861 |
var ringPip = smoothedLandmarks[14];
|
| 862 |
var pinkyTip = smoothedLandmarks[20];
|
| 863 |
var pinkyPip = smoothedLandmarks[18];
|
|
|
|
|
|
|
| 864 |
var middleExt = Math.max(0, Math.min(1, (middlePip.y - middleTip.y) / palmSize));
|
| 865 |
var ringExt = Math.max(0, Math.min(1, (ringPip.y - ringTip.y) / palmSize));
|
| 866 |
var pinkyExt = Math.max(0, Math.min(1, (pinkyPip.y - pinkyTip.y) / palmSize));
|
| 867 |
+
var indexExt = Math.max(0, Math.min(1, (smoothedLandmarks[6].y - smoothedLandmarks[8].y) / palmSize));
|
| 868 |
// Hand spread: average distance between adjacent fingertips / palm size
|
| 869 |
var tips = [smoothedLandmarks[4], smoothedLandmarks[8], middleTip, ringTip, pinkyTip];
|
| 870 |
var spreadSum = 0;
|
|
|
|
| 875 |
}
|
| 876 |
var handSpread = Math.max(0, Math.min(1, (spreadSum / 4) / (palmSize * 2)));
|
| 877 |
_this1.musicManager.updateFingerExpression({
|
| 878 |
+
fingerDistances: fingerDistances,
|
| 879 |
middleFinger: middleExt,
|
| 880 |
ringFinger: ringExt,
|
| 881 |
pinkyFinger: pinkyExt,
|
| 882 |
handSpread: handSpread
|
| 883 |
});
|
| 884 |
// --- Gesture-Driven Music ---
|
|
|
|
| 885 |
var fingerStates = {
|
| 886 |
+
thumb: fingerDistances.thumb > 0.3,
|
| 887 |
index: indexExt > 0.3,
|
| 888 |
middle: middleExt > 0.3,
|
| 889 |
ring: ringExt > 0.3,
|
|
|
|
| 892 |
// Track previous states and compute velocities
|
| 893 |
if (!_this1._prevFingerStates) _this1._prevFingerStates = {};
|
| 894 |
if (!_this1._prevHandPos) _this1._prevHandPos = {};
|
| 895 |
+
if (!_this1._prevFingerDistances) _this1._prevFingerDistances = {};
|
| 896 |
+
var prevFS = _this1._prevFingerStates[i] || { thumb: false, index: false, middle: false, ring: false, pinky: false };
|
| 897 |
var prevHP = _this1._prevHandPos[i] || { x: hand.anchorPos.x, y: hand.anchorPos.y };
|
| 898 |
+
var prevFD = _this1._prevFingerDistances[i] || { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
|
| 899 |
var handVelX = hand.anchorPos.x - prevHP.x;
|
| 900 |
var handVelY = hand.anchorPos.y - prevHP.y;
|
| 901 |
_this1.musicManager.updateGesture(i, {
|
| 902 |
fingerStates: fingerStates,
|
| 903 |
prevFingerStates: prevFS,
|
| 904 |
+
fingerDistances: fingerDistances,
|
| 905 |
+
prevFingerDistances: prevFD,
|
| 906 |
fingerExtensions: {
|
| 907 |
+
thumb: fingerDistances.thumb,
|
| 908 |
index: indexExt,
|
| 909 |
middle: middleExt,
|
| 910 |
ring: ringExt,
|
| 911 |
pinky: pinkyExt
|
| 912 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 913 |
handVelocity: { x: handVelX, y: handVelY },
|
| 914 |
+
handSpread: handSpread,
|
| 915 |
rootNote: note,
|
| 916 |
volume: velocity
|
| 917 |
});
|
| 918 |
_this1._prevFingerStates[i] = fingerStates;
|
| 919 |
_this1._prevHandPos[i] = { x: hand.anchorPos.x, y: hand.anchorPos.y };
|
| 920 |
+
_this1._prevFingerDistances[i] = Object.assign({}, fingerDistances);
|
| 921 |
} else {
|
| 922 |
// If it is a fist, make sure the arpeggio is stopped
|
| 923 |
_this1.musicManager.stopArpeggio(i);
|
|
|
|
| 930 |
});
|
| 931 |
// Per-finger drum sounds via updateDrumGesture
|
| 932 |
var drumWrist = smoothedLandmarks[0];
|
| 933 |
+
var drumPalmCenter = smoothedLandmarks[9];
|
| 934 |
+
var drumPalmDist = Math.sqrt(Math.pow(drumWrist.x - drumPalmCenter.x, 2) + Math.pow(drumWrist.y - drumPalmCenter.y, 2)) || 0.05;
|
| 935 |
+
var drumFingerTips = {
|
| 936 |
+
thumb: smoothedLandmarks[4],
|
| 937 |
+
index: smoothedLandmarks[8],
|
| 938 |
+
middle: smoothedLandmarks[12],
|
| 939 |
+
ring: smoothedLandmarks[16],
|
| 940 |
+
pinky: smoothedLandmarks[20]
|
| 941 |
+
};
|
| 942 |
+
var drumFingerDistances = {};
|
| 943 |
+
for (var dfname in drumFingerTips) {
|
| 944 |
+
var dtip = drumFingerTips[dfname];
|
| 945 |
+
var dfdx = dtip.x - drumPalmCenter.x;
|
| 946 |
+
var dfdy = dtip.y - drumPalmCenter.y;
|
| 947 |
+
var dfdist = Math.sqrt(dfdx * dfdx + dfdy * dfdy);
|
| 948 |
+
drumFingerDistances[dfname] = Math.max(0, Math.min(1, dfdist / (drumPalmDist * 2)));
|
| 949 |
+
}
|
| 950 |
if (_this1.musicManager.updateDrumGesture) {
|
| 951 |
_this1.musicManager.updateDrumGesture({
|
| 952 |
fingerStates: fingerStates,
|
| 953 |
+
fingerDistances: drumFingerDistances,
|
| 954 |
fingerExtensions: {
|
| 955 |
+
thumb: drumFingerDistances.thumb,
|
| 956 |
+
index: drumFingerDistances.index,
|
| 957 |
+
middle: drumFingerDistances.middle,
|
| 958 |
+
ring: drumFingerDistances.ring,
|
| 959 |
+
pinky: drumFingerDistances.pinky
|
| 960 |
}
|
| 961 |
});
|
| 962 |
}
|