Solshine commited on
Commit
19a0b4c
·
verified ·
1 Parent(s): d40d0f1

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. .claude/handoff.md +94 -0
  2. .claude/vision-reference.md +111 -0
  3. MusicManager.js +268 -248
  4. game.js +57 -21
.claude/handoff.md ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Hyperspace Jam — Session Handoff (2026-03-23)
2
+
3
+ ## STATUS: Working, needs per-finger sound refinement
4
+
5
+ Core app is stable: hand tracking, pad drone, sub-bass, drums, psychedelic visuals all work.
6
+ Per-finger sounds exist in code but need testing/tuning after the rampTo crash fix.
7
+
8
+ ## Links
9
+ - **HF Space**: https://huggingface.co/spaces/Solshine/hyperspace-jam
10
+ - **Direct URL**: https://solshine-hyperspace-jam.static.hf.space/index.html
11
+ - **GitHub**: https://github.com/SolshineCode/hyperspace-jam-v2
12
+
13
+ ## Deploy Command
14
+ ```bash
15
+ cd /c/Users/caleb/hyperspace-jam-v2-clean
16
+ cp /c/Users/caleb/hyperspace-jam-v2/*.js /c/Users/caleb/hyperspace-jam-v2/*.html /c/Users/caleb/hyperspace-jam-v2/*.css .
17
+ python -c "from huggingface_hub import HfApi; HfApi().upload_folder(folder_path='.', repo_id='Solshine/hyperspace-jam', repo_type='space', ignore_patterns=['.git*'])"
18
+ ```
19
+
20
+ ## What Works
21
+ - Webcam + MediaPipe hand tracking (2 hands) in HF Spaces
22
+ - Pad drone with harmony voice (root + detuned 5th), smooth pitch glide
23
+ - Sub-bass layer with wobble LFO (hand spread = wobble speed)
24
+ - Hand height (Y) → pitch across 5 octaves (C1 sub-bass to F5)
25
+ - Thumb-index pinch → volume (normalized by palm size, works at any distance)
26
+ - Proximity to camera → lowpass filter sweep + reverb/delay wet
27
+ - Fist gesture → cycle pad presets (Hypnotic Sub / Acid Growl / Trance Wash)
28
+ - Spacebar PANIC → kills all sound + 1 second mute window
29
+ - Hand 2 drums via DrumManager (index=kick, middle=snare, ring=hihat, pinky=clap)
30
+ - Psychedelic hue-rotating webcam filter (fast, saturated)
31
+ - SVG turbulence displacement (organic per-pixel warping)
32
+ - Smoke/wave CSS distortion
33
+ - Poincaré hyperbolic shader background (audio-reactive, breathing)
34
+ - Mandala geometry (pentagram, rays, rings between fingers)
35
+ - Shape pinch detection + tessellation shader
36
+ - Per-finger labels on both hands
37
+ - Controls hint panel (frosted glass, bottom center)
38
+ - Pre-allocated geometry pools (ShapeManager + MandalaVisualizer) for stability
39
+
40
+ ## PRIORITY 1: Per-Finger Sound Refinement
41
+
42
+ ### The root cause we found
43
+ A `Tone.js rampTo()` with near-zero values threw `RangeError` EVERY FRAME, which crashed `_updateHands()`, which killed ALL tracking AND sound. Fixed by removing the offending rampTo and wrapping in try/catch.
44
+
45
+ ### Current finger synth architecture (MusicManager.js)
46
+ **Hand 1 (synth hand) finger synths — all pre-allocated in start():**
47
+ - INDEX: MonoSynth "Acid Squelch" (sawtooth + filter sweep) → connected to delay
48
+ - MIDDLE: FMSynth "Laser Zap" (modulationIndex=25, square+sawtooth) → delay
49
+ - RING: MetalSynth "Crystal Chime" (resonance=3000, 1.5 octaves) → delay
50
+ - PINKY: MembraneSynth "Sub Drop" (pitchDecay=0.3, 6 octaves) → limiter
51
+
52
+ **Hand 2 (drum hand) finger synths — pre-allocated in start():**
53
+ - INDEX: MembraneSynth kick (8 octaves) → limiter
54
+ - MIDDLE: NoiseSynth "Riser" through AutoFilter → delay
55
+ - RING: NoiseSynth "Stutter" (rapid 4x bursts) → limiter
56
+ - PINKY: NoiseSynth "Crash" into dedicated Reverb(decay=8) → limiter
57
+
58
+ ### What needs work next session
59
+ 1. **Test with the crash fix** — the rampTo fix may have unblocked everything
60
+ 2. **Verify finger extension values** — add console.log in updateGesture to see actual ext values
61
+ 3. **The curl→extend threshold (0.25→0.35)** may need adjustment based on actual data
62
+ 4. **Consider going back to the arpeggiator pattern** as the user liked — add finger sounds ON TOP of the arp, not replacing it
63
+ 5. **The original arpeggiator's finger detection worked** because it used game.js's `_getFingerStates()` which is proven. Our custom extension calculation may be wrong.
64
+
65
+ ### Key insight from user
66
+ "The arpeggiator worked so well though" — the working arpeggiator-based version had reliable hand detection because it didn't have per-frame errors crashing the update loop. With the crash fixed, the current version should track properly. But if finger detection is still unreliable, consider using `_getFingerStates()` (boolean up/down, proven to work) instead of continuous extension values.
67
+
68
+ ## Key Files
69
+ | File | Purpose | Status |
70
+ |------|---------|--------|
71
+ | game.js | Main orchestrator (TRANSPILED ~1500 lines) | Many surgical edits, fragile |
72
+ | MusicManager.js | 10-finger psybass EDM engine | Crash fixed, needs testing |
73
+ | DrumManager.js | Drum sequencer | Stable, unchanged |
74
+ | WaveformVisualizer.js | Poincaré shader + breathing | Working |
75
+ | MandalaVisualizer.js | Sacred geometry | Pre-allocated pools, stable |
76
+ | ShapeManager.js | Pinch shapes + tessellation | Pre-allocated pools, stable |
77
+ | ShapeTessellationShader.js | GLSL for shape interiors | `ctr` fix applied |
78
+ | DisplacementFilter.js | SVG turbulence + smoke waves | Working |
79
+ | index.html | UI + controls hint | Updated labels |
80
+ | styles.css | Dark kiosk styling | Working |
81
+
82
+ ## Effects Chain
83
+ ```
84
+ Pad + Harmony → Filter → Chorus → Reverb → Limiter → Destination
85
+ Finger synths → Delay → Reverb → Limiter → Destination
86
+ Sub-bass → Limiter → Destination (direct, no effects)
87
+ Percussion → Limiter → Destination (direct, punchy)
88
+ Reverb → Analyser (for visualization)
89
+ ```
90
+
91
+ ## Remaining Phase 3 Features
92
+ - Phase 3.6: Multiplayer jam session (numHands: 4, treble+bass split)
93
+ - Dynamic shape internal tessellation (ShapeTessellationShader exists but untested)
94
+ - Attract mode logic (30s timer)
.claude/vision-reference.md ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Hyperspace Jam — Vision Reference: Dynamic Geometric Shapes with Internal Hyperbolic Tessellation
2
+
3
+ ## Source
4
+ Instagram reel: https://www.instagram.com/reel/DTYN4uwCtuG/
5
+ Described by video analysis agent 2026-03-22.
6
+
7
+ ## Core Interaction Model
8
+
9
+ ### Dynamic Anchor Points
10
+ - **Pinch gesture** (thumb + index touching) = single anchor point
11
+ - **2 anchors** = line (two parallel lines connecting points)
12
+ - **3 anchors** = triangle
13
+ - **4 anchors** = quadrilateral
14
+ - System smoothly transitions between states as anchors are added/removed
15
+
16
+ ### Anchor Sources
17
+ - Thumb tip (landmark 4)
18
+ - Index tip (landmark 8)
19
+ - Middle tip (landmark 12)
20
+ - Pinch = thumb+index merged into one point
21
+ - Both hands can contribute anchors simultaneously
22
+
23
+ ### Shape Rendering (Layer 1: Wireframe)
24
+ - **Nodes**: Small hollow white squares with solid white dot center
25
+ - **Edges**: Crisp solid white lines connecting nodes
26
+ - **Z-depth**: Line thickness and node size scale inversely with distance from camera
27
+ - Close to camera = thick lines, large nodes
28
+ - Far from camera = thin lines, small nodes
29
+ - No physics/gravity — pure tension between tracked points
30
+
31
+ ## Internal Visualization (Layer 2: Hyperbolic Geometry)
32
+
33
+ ### When a closed shape forms (held > ~0.2s):
34
+ - Internal space fills with **Poincaré disk tessellation**
35
+ - Central focal point inside the shape
36
+ - Tessellating shapes get smaller and denser toward edges (hyperbolic compression)
37
+ - Pattern dynamically re-tessellates as shape morphs with finger movement
38
+
39
+ ### Shape-specific tessellations:
40
+ - **Line**: 1D hyperbolic waves and interference rings between parallel lines
41
+ - **Triangle**: Triangle-based Poincaré tessellation with central singularity
42
+ - **Quadrilateral**: Quad-based tiling (e.g., {4,5} non-Euclidean tiling)
43
+
44
+ ### Dynamic behavior:
45
+ - Pattern scales and re-tessellates in real-time as fingers move
46
+ - Stretching fingers = tessellation scales, colors intensify
47
+ - Collapsing shape = reverse singularity wink-out effect
48
+
49
+ ## Psychedelic Coloring (Layer 3: DMT-Chrome)
50
+
51
+ ### Color palette:
52
+ - Deep blues, electric purples, neon pinks, vibrant greens
53
+ - Iridescent shifting between colors (not static)
54
+ - Colors flow along curved non-Euclidean geometry toward focal point
55
+
56
+ ### Effects:
57
+ - Biological pulsation rhythm (edges flare brighter at peaks)
58
+ - Fractal particles drift from nodes during transitions
59
+ - Maximum stretch = most intense iridescence
60
+ - Close to camera = overwhelming magenta/turquoise flares
61
+
62
+ ## Audio Hooks
63
+ - Shape proximity to camera → distortion/low-pass filter
64
+ - Close = distorted/filtered, far = clear
65
+ - Pattern complexity peak synced with audio distortion peak
66
+ - Pulsation rhythm could sync with beat
67
+
68
+ ## Multiplayer Jam Session (3+ hands detected)
69
+
70
+ ### Core Concept
71
+ When MediaPipe detects more than 2 hands, the system enters **multiplayer jam mode**. Each player pair gets distinct musical roles that complement each other — not just duplicates.
72
+
73
+ ### Player Role Assignment
74
+ - **Player 1 (hands 0-1)**: Treble / Lead — higher octave range (C3-C6), brighter synth presets, faster arpeggio patterns
75
+ - **Player 2 (hands 2-3)**: Bass / Foundation — lower octave range (C1-C3), deeper presets (Acid Bass, sub-sine), slower patterns, heavier reverb
76
+ - Could also split as: melodic vs rhythmic, or arp vs pad
77
+
78
+ ### Musical Separation Strategy
79
+ - Each player pair routes through its own PolySynth instance with distinct preset
80
+ - Separate effects chains (Player 1: bright delay, Player 2: deep reverb)
81
+ - Octave offset so they naturally harmonize rather than clash
82
+ - Same scale (C Minor Pentatonic) but different registers = instant musical compatibility
83
+ - BPM is shared — both players lock to the same Transport clock
84
+
85
+ ### Visual Differentiation
86
+ - Player 1 mandala/geometry: cyan + magenta palette
87
+ - Player 2 mandala/geometry: gold + green palette
88
+ - Internal Poincaré tessellations use different {P,Q} tilings per player
89
+ - Inter-player geometry: lines connecting across players' hands create larger mandala structures
90
+
91
+ ### Detection Logic
92
+ - MediaPipe HandLandmarker already supports `numHands: 4`
93
+ - Hands 0-1 = Player 1, Hands 2-3 = Player 2
94
+ - If only 2 hands detected, single-player mode (current behavior)
95
+ - If 3-4 hands detected, split into two players automatically
96
+ - UI indicator shows "JAM SESSION" when multiplayer is active
97
+
98
+ ### Scaling Considerations
99
+ - 4 hands = 84 landmarks at 30fps — still performant
100
+ - Two separate PolySynth instances may need careful volume balancing
101
+ - Master bus limiter (-2dB) protects speakers regardless of player count
102
+
103
+ ## Implementation Priority for Hyperspace Jam
104
+ 1. Pinch detection → merge thumb+index into single anchor
105
+ 2. Dynamic shape state (line → triangle → quad) based on active anchors
106
+ 3. Internal Poincaré tessellation shader filling the shape geometry
107
+ 4. DMT-chrome iridescent color flow
108
+ 5. Z-depth line thickness + node size scaling
109
+ 6. Audio filter modulation from shape proximity
110
+ 7. Pulsation and particle effects
111
+ 8. **Multiplayer jam session** — detect 3-4 hands, split into treble+bass roles
MusicManager.js CHANGED
@@ -1,20 +1,17 @@
1
  import * as Tone from 'https://esm.sh/tone';
2
 
3
- // Gesture-Driven Music Engine for Hyperspace Jam
4
- // Every finger on both hands produces a unique, radically different sound
5
  export class MusicManager {
6
  constructor() {
7
- // Synths
8
- this.padSynths = new Map(); // handId -> { synth, harmonySynth, currentRoot }
9
- this.activePatterns = this.padSynths; // backward compat — game.js checks .activePatterns.has(i)
10
 
11
- // Effects
12
  this.reverb = null;
13
  this.delay = null;
14
  this.chorus = null;
15
  this.analyser = null;
16
 
17
- // State tracking
18
  this.isStarted = false;
19
  this.handVolumes = new Map();
20
  this.fingerCooldowns = new Map();
@@ -24,191 +21,162 @@ export class MusicManager {
24
  this.PERC_COOLDOWN_MS = 150;
25
  this.HAND_VELOCITY_THRESHOLD = 0.15;
26
 
27
- // Finger -> semitone interval mapping for synth hand melodic triggers
28
  this.fingerIntervals = {
29
- index: 0, // root
30
- middle: 3, // minor third
31
- ring: 7, // perfect fifth
32
- pinky: 10 // minor seventh
33
  };
34
 
35
- // Extended C Minor Pentatonic scale
36
  this.scale = [
37
- 'C1', 'Eb1', 'G1', 'Bb1',
38
- 'C2', 'Eb2', 'F2', 'G2', 'Bb2',
39
- 'C3', 'Eb3', 'F3', 'G3', 'Bb3',
40
- 'C4', 'Eb4', 'F4', 'G4', 'Bb4',
41
- 'C5', 'Eb5', 'F5'
42
  ];
43
 
44
- // Pad timbre presets
45
  this.padPresets = [
46
- {
47
- name: 'Hypnotic Sub',
48
- oscillator: { type: 'sine' },
49
- envelope: { attack: 0.4, decay: 0.5, sustain: 0.6, release: 0.5 }
50
- },
51
- {
52
- name: 'Acid Growl',
53
- oscillator: { type: 'sawtooth' },
54
- envelope: { attack: 0.1, decay: 0.3, sustain: 0.5, release: 0.4 }
55
- },
56
- {
57
- name: 'Trance Wash',
58
- oscillator: { type: 'triangle' },
59
- envelope: { attack: 0.5, decay: 0.6, sustain: 0.55, release: 0.6 }
60
- }
61
  ];
62
  this.currentSynthIndex = 0;
63
 
64
- // Previous extension tracking for both hands
65
  this._prevExtensions = {};
66
- this._prevDrumExtensions = { index: 0, middle: 0, ring: 0, pinky: 0 };
67
- this._drumFingerCooldowns = { index: 0, middle: 0, ring: 0, pinky: 0 };
 
 
 
 
 
68
  }
69
 
70
  async start() {
71
  if (this.isStarted) return;
72
-
73
  await Tone.start();
74
 
75
- // === EFFECTS CHAIN ===
76
-
77
- // Master limiter
78
  this.limiter = new Tone.Limiter(-3).toDestination();
79
 
80
- // Dark cavernous reverb
81
- this.reverb = new Tone.Reverb({
82
- decay: 6,
83
- preDelay: 0.03,
84
- wet: 0.35
85
- }).connect(this.limiter);
86
 
87
- // Ping-pong delay for finger synths
88
- this.delay = new Tone.PingPongDelay({
89
- delayTime: '8n.',
90
- feedback: 0.35,
91
- wet: 0.2
92
- }).connect(this.reverb);
93
-
94
- // Chorus for pad width (no Phaser — CPU savings)
95
- this.chorus = new Tone.Chorus({
96
- frequency: 2,
97
- delayTime: 4,
98
- depth: 0.7
99
- }).connect(this.reverb);
100
  this.chorus.start();
101
 
102
- // Lowpass filter for proximity control (pad chain)
103
  this.filter = new Tone.Filter(16000, 'lowpass').connect(this.chorus);
104
 
105
- // Analyser for visualization — taps reverb output
106
  this.analyser = new Tone.Analyser('waveform', 1024);
107
  this.reverb.connect(this.analyser);
108
 
109
- // === HAND 1 FINGER SYNTHS (Synth Handi===0) ===
110
 
 
111
  this.fingerSynths = {};
112
-
113
- // INDEX: "Acid Squelch" — MonoSynth with sawtooth + filter sweep
114
  this.fingerSynths.index = new Tone.MonoSynth({
115
  oscillator: { type: 'sawtooth' },
116
- filter: {
117
- Q: 8,
118
- type: 'lowpass',
119
- rolloff: -24
120
- },
121
- envelope: { attack: 0.005, decay: 0.2, sustain: 0.1, release: 0.3 },
122
  filterEnvelope: {
123
- attack: 0.001,
124
- decay: 0.15,
125
- sustain: 0.05,
126
- release: 0.2,
127
- baseFrequency: 200,
128
- octaves: 4,
129
- exponent: 2
130
  }
131
  }).connect(this.delay);
132
  this.fingerSynths.index.volume.value = -4;
133
 
134
- // MIDDLE: "Laser Zap"FMSynth with extreme modulation, fast pitch sweep
135
  this.fingerSynths.middle = new Tone.FMSynth({
136
- harmonicity: 8,
137
- modulationIndex: 25,
138
  oscillator: { type: 'square' },
139
- envelope: { attack: 0.001, decay: 0.15, sustain: 0, release: 0.1 },
140
  modulation: { type: 'sawtooth' },
141
- modulationEnvelope: { attack: 0.001, decay: 0.08, sustain: 0, release: 0.05 }
142
  }).connect(this.delay);
143
  this.fingerSynths.middle.volume.value = -6;
144
 
145
- // RING: "Crystal Chime"MetalSynth, bright bell
146
  this.fingerSynths.ring = new Tone.MetalSynth({
147
- harmonicity: 12,
148
- modulationIndex: 24,
149
- resonance: 3000,
150
- octaves: 1.5,
151
  envelope: { attack: 0.001, decay: 0.4, sustain: 0, release: 0.3 }
152
  }).connect(this.delay);
153
  this.fingerSynths.ring.volume.value = -8;
154
 
155
- // PINKY: "Sub Drop"MembraneSynth with long pitch decay
 
156
  this.fingerSynths.pinky = new Tone.MembraneSynth({
157
- pitchDecay: 0.3,
158
- octaves: 6,
159
  oscillator: { type: 'sine' },
160
  envelope: { attack: 0.001, decay: 0.8, sustain: 0, release: 0.5 }
161
- }).connect(this.limiter); // sub direct — no delay/reverb muddying
162
  this.fingerSynths.pinky.volume.value = -2;
163
 
164
- // === HAND 2 FINGER SYNTHS (Drum Hand i===1) ===
 
 
 
 
 
 
 
 
 
 
 
165
 
 
166
  this.drumFingerSynths = {};
167
 
168
- // INDEX: Kick — deep membrane hit (same as before but dedicated to drum hand)
169
  this.drumFingerSynths.index = new Tone.MembraneSynth({
170
- pitchDecay: 0.08,
171
- octaves: 8,
172
  oscillator: { type: 'sine' },
173
  envelope: { attack: 0.001, decay: 0.5, sustain: 0, release: 0.5 }
174
- }).connect(this.limiter); // percussion direct
175
  this.drumFingerSynths.index.volume.value = -4;
176
 
177
- // MIDDLE: "Sci-Fi Riser" — noise with filter sweeping UP
178
  this.drumFingerSynths.middle = new Tone.NoiseSynth({
179
  noise: { type: 'pink' },
180
  envelope: { attack: 0.05, decay: 0.6, sustain: 0, release: 0.3 }
181
  });
182
  this._riserFilter = new Tone.AutoFilter({
183
- frequency: 4,
184
- baseFrequency: 200,
185
- octaves: 6,
186
  filter: { type: 'bandpass', Q: 2 }
187
  }).connect(this.delay);
188
  this._riserFilter.start();
189
  this.drumFingerSynths.middle.connect(this._riserFilter);
190
  this.drumFingerSynths.middle.volume.value = -8;
191
 
192
- // RING: "Granular Stutter" — rapid burst noise hits
193
  this.drumFingerSynths.ring = new Tone.NoiseSynth({
194
  noise: { type: 'white' },
195
  envelope: { attack: 0.001, decay: 0.02, sustain: 0, release: 0.01 }
196
- }).connect(this.limiter); // percussion direct, punchy
197
  this.drumFingerSynths.ring.volume.value = -10;
198
 
199
- // PINKY: "Reverb Crash" — noise hit into massive reverb
200
- this._crashReverb = new Tone.Reverb({
201
- decay: 8,
202
- preDelay: 0.01,
203
- wet: 0.9
204
- }).connect(this.limiter);
205
  this.drumFingerSynths.pinky = new Tone.NoiseSynth({
206
  noise: { type: 'white' },
207
  envelope: { attack: 0.001, decay: 0.15, sustain: 0, release: 0.1 }
208
  }).connect(this._crashReverb);
209
  this.drumFingerSynths.pinky.volume.value = -6;
210
 
211
- // Keep legacy references for backward compat
 
 
 
 
 
 
 
212
  this.kickSynth = this.drumFingerSynths.index;
213
  this.hatSynth = new Tone.NoiseSynth({
214
  noise: { type: 'white' },
@@ -217,33 +185,36 @@ export class MusicManager {
217
  this.hatSynth.volume.value = -12;
218
  this.pluckSynth = { releaseAll: () => {} };
219
 
220
- // === SUB-BASS (always-on low foundation) ===
221
  this.subBass = new Tone.Synth({
222
  oscillator: { type: 'sine' },
223
  envelope: { attack: 0.3, decay: 0, sustain: 1, release: 0.8 }
224
- }).connect(this.limiter); // sub direct
225
  this.subBass.volume.value = -18;
226
 
227
- // Sub-bass wobble LFO
228
- this.wobbleLFO = new Tone.LFO({
229
- frequency: 0.3,
230
- min: -22,
231
- max: -12
232
- });
233
  this.wobbleLFO.connect(this.subBass.volume);
234
  this.wobbleLFO.start();
235
 
 
 
 
236
  this.isStarted = true;
237
- console.log('Psychedelic Bass EDM engine ready — 10-finger unique sounds, gesture-driven.');
238
  }
239
 
240
- // --- Pad Management (Layer 1) ---
 
 
 
 
 
 
241
 
242
  startArpeggio(handId, rootNote) {
243
  if (!this.isStarted || this.padSynths.has(handId) || this._panicMuted) return;
244
 
245
  const preset = this.padPresets[this.currentSynthIndex];
246
-
247
  const pad = new Tone.Synth({
248
  oscillator: { ...preset.oscillator },
249
  envelope: { ...preset.envelope }
@@ -262,13 +233,19 @@ export class MusicManager {
262
  pad.triggerAttack(freq, Tone.now());
263
  harmonyPad.triggerAttack(freq * 1.498, Tone.now());
264
 
265
- if (this.subBass) {
266
- this.subBass.triggerAttack(freq / 2, Tone.now());
267
- }
268
 
269
  this.padSynths.set(handId, { synth: pad, harmonySynth: harmonyPad, currentRoot: rootNote });
270
  this.handVolumes.set(handId, 0.2);
271
- this.fingerCooldowns.set(handId, { index: 0, middle: 0, ring: 0, pinky: 0 });
 
 
 
 
 
 
 
 
272
  }
273
 
274
  updateArpeggio(handId, newRootNote) {
@@ -277,12 +254,14 @@ export class MusicManager {
277
 
278
  const freq = Tone.Frequency(newRootNote).toFrequency();
279
  padData.synth.frequency.rampTo(freq, 0.15);
280
- if (padData.harmonySynth) {
281
- padData.harmonySynth.frequency.rampTo(freq * 1.498, 0.15);
282
- }
283
- if (this.subBass) {
284
- this.subBass.frequency.rampTo(freq / 2, 0.2);
285
- }
 
 
286
  padData.currentRoot = newRootNote;
287
  }
288
 
@@ -294,9 +273,7 @@ export class MusicManager {
294
  this.handVolumes.set(handId, clamped);
295
  const db = -30 + clamped * 26;
296
  padData.synth.volume.rampTo(db, 0.1);
297
- if (padData.harmonySynth) {
298
- padData.harmonySynth.volume.rampTo(db - 6, 0.1);
299
- }
300
  }
301
 
302
  stopArpeggio(handId) {
@@ -308,13 +285,19 @@ export class MusicManager {
308
  this.subBass.triggerRelease(Tone.now());
309
  }
310
 
 
 
 
 
 
 
311
  if (!this._pendingDisposals) this._pendingDisposals = new Set();
312
  const synths = [padData.synth, padData.harmonySynth].filter(Boolean);
313
  for (const synth of synths) {
314
  if (!this._pendingDisposals.has(synth)) {
315
  this._pendingDisposals.add(synth);
316
  setTimeout(() => {
317
- try { synth.dispose(); } catch(e) { /* already disposed */ }
318
  this._pendingDisposals.delete(synth);
319
  }, 2000);
320
  }
@@ -325,41 +308,45 @@ export class MusicManager {
325
  }
326
  }
327
 
328
- // --- Gesture Processing: Synth Hand (hand 0) ---
329
- // HYBRID: triggers on curl→extend PLUS continuous modulation from extension amount
330
- // Every finger ALWAYS changes the sound when extended — impossible to not hear it
331
 
332
  updateGesture(handId, gestureData) {
333
  if (!this.isStarted || this._panicMuted) return;
334
 
335
  const { fingerStates, handVelocity, rootNote } = gestureData;
336
-
337
  const now = performance.now();
338
  const cooldowns = this.fingerCooldowns.get(handId);
339
  if (!cooldowns) return;
340
 
341
  if (!this._prevExtensions[handId]) {
342
- this._prevExtensions[handId] = { index: 0, middle: 0, ring: 0, pinky: 0 };
343
  }
344
  const prevExt = this._prevExtensions[handId];
345
 
346
- const ext = gestureData.fingerExtensions || {
347
- index: fingerStates.index ? 0.8 : 0.1,
348
- middle: fingerStates.middle ? 0.8 : 0.1,
349
- ring: fingerStates.ring ? 0.8 : 0.1,
350
- pinky: fingerStates.pinky ? 0.8 : 0.1
351
  };
352
 
 
 
 
 
 
 
353
  const rootFreq = Tone.Frequency(rootNote).toFrequency();
354
 
355
- // === INDEX: Acid Squelch — CONTINUOUS filter cutoff from extension ===
 
 
 
 
 
 
356
  if (this.fingerSynths.index) {
357
- try {
358
- const cutoff = 200 + ext.index * 4000;
359
- this.fingerSynths.index.filter.frequency.value = cutoff;
360
- } catch(e) {}
361
- // Trigger on curl→extend transition
362
- if (prevExt.index < 0.2 && ext.index > 0.35 && (now - cooldowns.index) > this.FINGER_COOLDOWN_MS) {
363
  cooldowns.index = now;
364
  this.fingerSynths.index.triggerAttackRelease(
365
  Tone.Frequency(rootFreq).toNote(), '4n', Tone.now(), 0.8
@@ -367,57 +354,99 @@ export class MusicManager {
367
  }
368
  }
369
 
370
- // === MIDDLE: Laser Zap — trigger only, no continuous volume ramp ===
 
 
 
371
  if (this.fingerSynths.middle) {
372
- // Trigger on transition
373
- if (prevExt.middle < 0.2 && ext.middle > 0.35 && (now - cooldowns.middle) > this.FINGER_COOLDOWN_MS) {
 
 
 
 
374
  cooldowns.middle = now;
375
- const zapFreq = rootFreq * Math.pow(2, 3/12); // minor 3rd
376
  this.fingerSynths.middle.triggerAttackRelease(
377
  Tone.Frequency(zapFreq).toNote(), '8n', Tone.now(), 0.9
378
  );
379
  }
380
  }
 
 
 
 
 
 
 
381
 
382
- // === RING: Crystal Chime — trigger only ===
 
 
 
383
  if (this.fingerSynths.ring) {
384
- // Trigger on transition
385
- if (prevExt.ring < 0.2 && ext.ring > 0.35 && (now - cooldowns.ring) > this.FINGER_COOLDOWN_MS) {
386
  cooldowns.ring = now;
387
- const chimeFreq = rootFreq * Math.pow(2, 7/12); // 5th
388
  this.fingerSynths.ring.triggerAttackRelease(chimeFreq, '4n', Tone.now(), 0.8);
389
  }
390
  }
 
 
 
 
 
 
 
 
 
391
 
392
- // === PINKY: Sub Drop — extension controls pitch bend depth ===
 
 
 
393
  if (this.fingerSynths.pinky) {
394
- // Trigger on transition always very audible
395
- if (prevExt.pinky < 0.2 && ext.pinky > 0.35 && (now - cooldowns.pinky) > this.FINGER_COOLDOWN_MS) {
396
  cooldowns.pinky = now;
397
  this.fingerSynths.pinky.triggerAttackRelease('C1', '4n', Tone.now(), 0.9);
398
  }
399
  }
 
 
 
 
 
 
 
400
 
401
- // === CONTINUOUS: Delay wet amount from average finger extension ===
402
- try {
403
- const avgExt = (ext.index + ext.middle + ext.ring + ext.pinky) / 4;
404
- if (this.delay && isFinite(avgExt)) {
405
- this.delay.wet.value = 0.05 + Math.max(0, Math.min(1, avgExt)) * 0.4;
406
- }
407
- } catch(e) {}
 
 
 
 
 
 
 
 
408
 
409
  // Store for next frame
410
- prevExt.index = ext.index;
411
- prevExt.middle = ext.middle;
412
- prevExt.ring = ext.ring;
413
- prevExt.pinky = ext.pinky;
 
414
 
415
  // Percussive hits on sharp hand movement
416
  if ((now - this.percCooldown) > this.PERC_COOLDOWN_MS) {
417
  const vx = handVelocity?.x || 0;
418
  const vy = handVelocity?.y || 0;
419
  const magnitude = Math.sqrt(vx * vx + vy * vy);
420
-
421
  if (magnitude > this.HAND_VELOCITY_THRESHOLD) {
422
  this.percCooldown = now;
423
  if (vy > this.HAND_VELOCITY_THRESHOLD) {
@@ -429,8 +458,7 @@ export class MusicManager {
429
  }
430
  }
431
 
432
- // --- Gesture Processing: Drum Hand (hand 1) ---
433
- // HYBRID: triggers PLUS continuous modulation — every finger always does something
434
 
435
  updateDrumGesture(gestureData) {
436
  if (!this.isStarted || this._panicMuted) return;
@@ -439,96 +467,100 @@ export class MusicManager {
439
  const prevExt = this._prevDrumExtensions;
440
  const cooldowns = this._drumFingerCooldowns;
441
 
442
- const fingerStates = gestureData.fingerStates || {};
443
- const ext = gestureData.fingerExtensions || {
444
- index: fingerStates.index ? 0.8 : 0.1,
445
- middle: fingerStates.middle ? 0.8 : 0.1,
446
- ring: fingerStates.ring ? 0.8 : 0.1,
447
- pinky: fingerStates.pinky ? 0.8 : 0.1
448
  };
449
 
450
- // === INDEX: Kick — trigger + continuous volume ===
 
 
 
 
 
 
 
 
 
 
 
 
 
 
451
  if (this.drumFingerSynths.index) {
452
- if (prevExt.index < 0.2 && ext.index > 0.35 && (now - cooldowns.index) > this.FINGER_COOLDOWN_MS) {
 
 
 
 
453
  cooldowns.index = now;
454
  this.drumFingerSynths.index.triggerAttackRelease('C1', '8n', Tone.now(), 0.9);
455
  }
456
  }
457
 
458
- // === MIDDLE: Riser — continuous: extension controls filter sweep position ===
459
  if (this._riserFilter) {
460
- this._riserFilter.baseFrequency = 100 + ext.middle * 3000;
 
461
  }
462
  if (this.drumFingerSynths.middle) {
463
- if (prevExt.middle < 0.2 && ext.middle > 0.35 && (now - cooldowns.middle) > 200) {
464
  cooldowns.middle = now;
465
  this.drumFingerSynths.middle.triggerAttackRelease('4n', Tone.now(), 0.7);
466
  }
467
  }
468
 
469
- // === RING: Stutter — trigger rapid hits ===
470
  if (this.drumFingerSynths.ring) {
471
- if (prevExt.ring < 0.2 && ext.ring > 0.35 && (now - cooldowns.ring) > 250) {
472
  cooldowns.ring = now;
473
- for (let i = 0; i < 4; i++) {
 
 
474
  setTimeout(() => {
475
  try { this.drumFingerSynths.ring.triggerAttackRelease('64n', Tone.now(), 0.6); } catch(e) {}
476
- }, i * 45);
477
  }
478
  }
479
  }
480
 
481
- // === PINKY: Reverb Crash — trigger + continuous reverb tail amount ===
482
  if (this._crashReverb) {
483
- this._crashReverb.wet.value = 0.5 + ext.pinky * 0.5;
484
  }
485
  if (this.drumFingerSynths.pinky) {
486
- if (prevExt.pinky < 0.2 && ext.pinky > 0.35 && (now - cooldowns.pinky) > 300) {
487
  cooldowns.pinky = now;
488
  this.drumFingerSynths.pinky.triggerAttackRelease('8n', Tone.now(), 0.8);
489
  }
490
  }
491
 
492
- // === CONTINUOUS: chorus depth from average drum hand extension ===
493
- const avgDrumExt = (ext.index + ext.middle + ext.ring + ext.pinky) / 4;
494
  if (this.chorus) {
495
- this.chorus.depth = 0.3 + avgDrumExt * 0.7;
496
  }
497
 
498
- prevExt.index = ext.index;
499
- prevExt.middle = ext.middle;
500
- prevExt.ring = ext.ring;
501
- prevExt.pinky = ext.pinky;
 
502
  }
503
 
504
- // --- Finger Expression (effects modulation) ---
505
 
506
  updateFingerExpression(params) {
507
  if (!this.isStarted) return;
508
 
509
- const { middleFinger, ringFinger, pinkyFinger, handSpread } = params;
510
-
511
- // Ring finger -> delay feedback
512
- if (this.delay) {
513
- this.delay.feedback.value = 0.15 + ringFinger * 0.45;
514
- }
515
-
516
- // Pinky -> reverb depth
517
- if (this.reverb) {
518
- this.reverb.wet.value = 0.2 + pinkyFinger * 0.4;
519
- }
520
 
521
- // Hand spread -> wobble LFO speed
522
  if (this.wobbleLFO) {
523
  this.wobbleLFO.frequency.value = 0.2 + handSpread * 6;
524
  }
525
-
526
- // Hand spread -> pad detune
527
  this.padSynths.forEach(padData => {
528
  padData.synth.detune.rampTo(handSpread * 40, 0.1);
529
- if (padData.harmonySynth) {
530
- padData.harmonySynth.detune.rampTo(-handSpread * 25, 0.1);
531
- }
532
  });
533
  }
534
 
@@ -564,23 +596,17 @@ export class MusicManager {
564
  const cutoff = 16000 * Math.pow(0.075, value);
565
  this.filter.frequency.rampTo(cutoff, 0.2);
566
  }
567
- if (this.reverb) {
568
- this.reverb.wet.value = 0.3 + value * 0.3;
569
- }
570
- if (this.delay) {
571
- this.delay.wet.value = 0.15 + value * 0.2;
572
- }
573
- if (this.chorus) {
574
- this.chorus.depth = 0.4 + value * 0.4;
575
- }
576
  }
577
 
578
- // --- PANIC: kill ALL sound immediately ---
 
579
  panic() {
580
  this._panicMuted = true;
581
  setTimeout(() => { this._panicMuted = false; }, 1000);
582
 
583
- // Stop all pads
584
  this.padSynths.forEach((padData) => {
585
  try { padData.synth.triggerRelease(Tone.now()); } catch(e) {}
586
  try { if (padData.harmonySynth) padData.harmonySynth.triggerRelease(Tone.now()); } catch(e) {}
@@ -593,44 +619,38 @@ export class MusicManager {
593
  this.handVolumes.clear();
594
  this.fingerCooldowns.clear();
595
 
596
- // Stop sub-bass
597
- if (this.subBass) {
598
- try { this.subBass.triggerRelease(Tone.now()); } catch(e) {}
599
- }
 
600
 
601
- // Silence all synth hand finger synths
602
  if (this.fingerSynths) {
603
  for (const finger of ['index', 'middle', 'ring', 'pinky']) {
604
  try { this.fingerSynths[finger].triggerRelease(Tone.now()); } catch(e) {}
605
  }
606
  }
607
-
608
- // Silence all drum hand finger synths
609
  if (this.drumFingerSynths) {
610
  for (const finger of ['index', 'middle', 'ring', 'pinky']) {
611
  try { this.drumFingerSynths[finger].triggerRelease(Tone.now()); } catch(e) {}
612
  }
613
  }
614
 
615
- // Kill wobble LFO temporarily
616
  if (this.wobbleLFO) {
617
  this.wobbleLFO.stop();
618
  setTimeout(() => { try { this.wobbleLFO.start(); } catch(e) {} }, 1000);
619
  }
620
 
621
- // Reset filter
622
- if (this.filter) {
623
- this.filter.frequency.value = 16000;
624
- }
625
-
626
- // Reset effect wet levels
627
  if (this.reverb) this.reverb.wet.value = 0.35;
628
  if (this.delay) this.delay.wet.value = 0.2;
629
  if (this.chorus) this.chorus.depth = 0.6;
 
630
 
631
- // Reset drum extension tracking
632
- this._prevDrumExtensions = { index: 0, middle: 0, ring: 0, pinky: 0 };
633
- this._drumFingerCooldowns = { index: 0, middle: 0, ring: 0, pinky: 0 };
 
634
 
635
  console.log('PANIC — all sound killed, muted for 1 second');
636
  }
 
1
  import * as Tone from 'https://esm.sh/tone';
2
 
3
+ // Psychedelic EDM Synth Engine Comprehensive Hand-Distance Control
4
+ // Every finger's distance from palm continuously drives dramatic synth parameters
5
  export class MusicManager {
6
  constructor() {
7
+ this.padSynths = new Map();
8
+ this.activePatterns = this.padSynths;
 
9
 
 
10
  this.reverb = null;
11
  this.delay = null;
12
  this.chorus = null;
13
  this.analyser = null;
14
 
 
15
  this.isStarted = false;
16
  this.handVolumes = new Map();
17
  this.fingerCooldowns = new Map();
 
21
  this.PERC_COOLDOWN_MS = 150;
22
  this.HAND_VELOCITY_THRESHOLD = 0.15;
23
 
 
24
  this.fingerIntervals = {
25
+ index: 0, middle: 3, ring: 7, pinky: 10
 
 
 
26
  };
27
 
28
+ // Extended C Minor Pentatonic
29
  this.scale = [
30
+ 'C1','Eb1','G1','Bb1','C2','Eb2','F2','G2','Bb2',
31
+ 'C3','Eb3','F3','G3','Bb3','C4','Eb4','F4','G4','Bb4','C5','Eb5','F5'
 
 
 
32
  ];
33
 
 
34
  this.padPresets = [
35
+ { name: 'Hypnotic Sub', oscillator: { type: 'sine' }, envelope: { attack: 0.4, decay: 0.5, sustain: 0.6, release: 0.5 } },
36
+ { name: 'Acid Growl', oscillator: { type: 'sawtooth' }, envelope: { attack: 0.1, decay: 0.3, sustain: 0.5, release: 0.4 } },
37
+ { name: 'Trance Wash', oscillator: { type: 'triangle' }, envelope: { attack: 0.5, decay: 0.6, sustain: 0.55, release: 0.6 } },
38
+ { name: 'PWM Swirl', oscillator: { type: 'pwm', modulationFrequency: 0.5 }, envelope: { attack: 0.3, decay: 0.4, sustain: 0.7, release: 0.6 } },
39
+ { name: 'Fat Square', oscillator: { type: 'fatsquare', spread: 30 }, envelope: { attack: 0.2, decay: 0.3, sustain: 0.6, release: 0.5 } }
 
 
 
 
 
 
 
 
 
 
40
  ];
41
  this.currentSynthIndex = 0;
42
 
 
43
  this._prevExtensions = {};
44
+ this._prevDrumExtensions = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
45
+ this._drumFingerCooldowns = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
46
+
47
+ // Smoothed finger distances for continuous control (prevents jitter)
48
+ this._smoothDist = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
49
+ this._smoothDrumDist = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
50
+ this._SMOOTH_ALPHA = 0.25; // smoothing factor
51
  }
52
 
53
  async start() {
54
  if (this.isStarted) return;
 
55
  await Tone.start();
56
 
57
+ // === MASTER EFFECTS CHAIN ===
 
 
58
  this.limiter = new Tone.Limiter(-3).toDestination();
59
 
60
+ // Dark reverb
61
+ this.reverb = new Tone.Reverb({ decay: 6, preDelay: 0.03, wet: 0.35 }).connect(this.limiter);
62
+
63
+ // Ping-pong delay
64
+ this.delay = new Tone.PingPongDelay({ delayTime: '8n.', feedback: 0.35, wet: 0.2 }).connect(this.reverb);
 
65
 
66
+ // Chorus
67
+ this.chorus = new Tone.Chorus({ frequency: 2, delayTime: 4, depth: 0.7 }).connect(this.reverb);
 
 
 
 
 
 
 
 
 
 
 
68
  this.chorus.start();
69
 
70
+ // Master proximity filter
71
  this.filter = new Tone.Filter(16000, 'lowpass').connect(this.chorus);
72
 
73
+ // Analyser for visualization
74
  this.analyser = new Tone.Analyser('waveform', 1024);
75
  this.reverb.connect(this.analyser);
76
 
77
+ // NOTE: Thumb + Index are volume control (pinch) no continuous effects on them
78
 
79
+ // === INDEX: Acid Squelch — continuously pitched, filter sweeps with distance ===
80
  this.fingerSynths = {};
 
 
81
  this.fingerSynths.index = new Tone.MonoSynth({
82
  oscillator: { type: 'sawtooth' },
83
+ filter: { Q: 8, type: 'lowpass', rolloff: -24 },
84
+ envelope: { attack: 0.005, decay: 0.2, sustain: 0.3, release: 0.3 },
 
 
 
 
85
  filterEnvelope: {
86
+ attack: 0.001, decay: 0.15, sustain: 0.1, release: 0.2,
87
+ baseFrequency: 200, octaves: 4, exponent: 2
 
 
 
 
 
88
  }
89
  }).connect(this.delay);
90
  this.fingerSynths.index.volume.value = -4;
91
 
92
+ // === MIDDLE: FM Chaosdistance controls modulation index (clean → insane) ===
93
  this.fingerSynths.middle = new Tone.FMSynth({
94
+ harmonicity: 3,
95
+ modulationIndex: 1,
96
  oscillator: { type: 'square' },
97
+ envelope: { attack: 0.01, decay: 0.3, sustain: 0.2, release: 0.3 },
98
  modulation: { type: 'sawtooth' },
99
+ modulationEnvelope: { attack: 0.01, decay: 0.15, sustain: 0.1, release: 0.1 }
100
  }).connect(this.delay);
101
  this.fingerSynths.middle.volume.value = -6;
102
 
103
+ // === RING: Shimmer Delaydistance controls delay time + feedback (tight → infinite) ===
104
  this.fingerSynths.ring = new Tone.MetalSynth({
105
+ harmonicity: 12, modulationIndex: 24,
106
+ resonance: 3000, octaves: 1.5,
 
 
107
  envelope: { attack: 0.001, decay: 0.4, sustain: 0, release: 0.3 }
108
  }).connect(this.delay);
109
  this.fingerSynths.ring.volume.value = -8;
110
 
111
+ // === PINKY: Sub Drop + Bitcrusher distance controls crush depth ===
112
+ this.bitcrusher = new Tone.BitCrusher({ bits: 16 }).connect(this.limiter);
113
  this.fingerSynths.pinky = new Tone.MembraneSynth({
114
+ pitchDecay: 0.3, octaves: 6,
 
115
  oscillator: { type: 'sine' },
116
  envelope: { attack: 0.001, decay: 0.8, sustain: 0, release: 0.5 }
117
+ }).connect(this.bitcrusher);
118
  this.fingerSynths.pinky.volume.value = -2;
119
 
120
+ // === CONTINUOUS SYNTHS (always sounding when hand present) ===
121
+ // Only on middle/ring/pinky — NOT thumb/index (those are volume fingers)
122
+
123
+ // Middle continuous: FM pad that gets chaotic with distance
124
+ this.middleContinuous = new Tone.FMSynth({
125
+ harmonicity: 2, modulationIndex: 0.5,
126
+ oscillator: { type: 'sine' },
127
+ envelope: { attack: 0.4, decay: 0, sustain: 1, release: 0.6 },
128
+ modulation: { type: 'triangle' },
129
+ modulationEnvelope: { attack: 0.4, decay: 0, sustain: 1, release: 0.6 }
130
+ }).connect(this.chorus);
131
+ this.middleContinuous.volume.value = -20;
132
 
133
+ // === DRUM HAND SYNTHS (hand 1) ===
134
  this.drumFingerSynths = {};
135
 
136
+ // Index: Kick
137
  this.drumFingerSynths.index = new Tone.MembraneSynth({
138
+ pitchDecay: 0.08, octaves: 8,
 
139
  oscillator: { type: 'sine' },
140
  envelope: { attack: 0.001, decay: 0.5, sustain: 0, release: 0.5 }
141
+ }).connect(this.limiter);
142
  this.drumFingerSynths.index.volume.value = -4;
143
 
144
+ // Middle: Sci-Fi Riser
145
  this.drumFingerSynths.middle = new Tone.NoiseSynth({
146
  noise: { type: 'pink' },
147
  envelope: { attack: 0.05, decay: 0.6, sustain: 0, release: 0.3 }
148
  });
149
  this._riserFilter = new Tone.AutoFilter({
150
+ frequency: 4, baseFrequency: 200, octaves: 6,
 
 
151
  filter: { type: 'bandpass', Q: 2 }
152
  }).connect(this.delay);
153
  this._riserFilter.start();
154
  this.drumFingerSynths.middle.connect(this._riserFilter);
155
  this.drumFingerSynths.middle.volume.value = -8;
156
 
157
+ // Ring: Granular Stutter
158
  this.drumFingerSynths.ring = new Tone.NoiseSynth({
159
  noise: { type: 'white' },
160
  envelope: { attack: 0.001, decay: 0.02, sustain: 0, release: 0.01 }
161
+ }).connect(this.limiter);
162
  this.drumFingerSynths.ring.volume.value = -10;
163
 
164
+ // Pinky: Reverb Crash
165
+ this._crashReverb = new Tone.Reverb({ decay: 8, preDelay: 0.01, wet: 0.9 }).connect(this.limiter);
 
 
 
 
166
  this.drumFingerSynths.pinky = new Tone.NoiseSynth({
167
  noise: { type: 'white' },
168
  envelope: { attack: 0.001, decay: 0.15, sustain: 0, release: 0.1 }
169
  }).connect(this._crashReverb);
170
  this.drumFingerSynths.pinky.volume.value = -6;
171
 
172
+ // Thumb: Drum hand thumb = tempo wobble synth
173
+ this.drumThumbSynth = new Tone.Synth({
174
+ oscillator: { type: 'sine' },
175
+ envelope: { attack: 0.01, decay: 0.3, sustain: 0, release: 0.2 }
176
+ }).connect(this.limiter);
177
+ this.drumThumbSynth.volume.value = -10;
178
+
179
+ // Legacy references
180
  this.kickSynth = this.drumFingerSynths.index;
181
  this.hatSynth = new Tone.NoiseSynth({
182
  noise: { type: 'white' },
 
185
  this.hatSynth.volume.value = -12;
186
  this.pluckSynth = { releaseAll: () => {} };
187
 
188
+ // === SUB-BASS ===
189
  this.subBass = new Tone.Synth({
190
  oscillator: { type: 'sine' },
191
  envelope: { attack: 0.3, decay: 0, sustain: 1, release: 0.8 }
192
+ }).connect(this.limiter);
193
  this.subBass.volume.value = -18;
194
 
195
+ this.wobbleLFO = new Tone.LFO({ frequency: 0.3, min: -22, max: -12 });
 
 
 
 
 
196
  this.wobbleLFO.connect(this.subBass.volume);
197
  this.wobbleLFO.start();
198
 
199
+ // Track continuous synth state
200
+ this._continuousActive = false;
201
+
202
  this.isStarted = true;
203
+ console.log('Psychedelic EDM engine v2 ready — 10-finger distance-from-palm control');
204
  }
205
 
206
+ // --- Smoothing helper ---
207
+ _smooth(target, key, raw) {
208
+ target[key] += (raw - target[key]) * this._SMOOTH_ALPHA;
209
+ return target[key];
210
+ }
211
+
212
+ // --- Pad Management ---
213
 
214
  startArpeggio(handId, rootNote) {
215
  if (!this.isStarted || this.padSynths.has(handId) || this._panicMuted) return;
216
 
217
  const preset = this.padPresets[this.currentSynthIndex];
 
218
  const pad = new Tone.Synth({
219
  oscillator: { ...preset.oscillator },
220
  envelope: { ...preset.envelope }
 
233
  pad.triggerAttack(freq, Tone.now());
234
  harmonyPad.triggerAttack(freq * 1.498, Tone.now());
235
 
236
+ if (this.subBass) this.subBass.triggerAttack(freq / 2, Tone.now());
 
 
237
 
238
  this.padSynths.set(handId, { synth: pad, harmonySynth: harmonyPad, currentRoot: rootNote });
239
  this.handVolumes.set(handId, 0.2);
240
+ this.fingerCooldowns.set(handId, { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 });
241
+
242
+ // Start continuous synths (middle only — thumb/index are volume fingers)
243
+ if (!this._continuousActive) {
244
+ this._continuousActive = true;
245
+ try {
246
+ this.middleContinuous.triggerAttack(freq * 1.498, Tone.now());
247
+ } catch(e) {}
248
+ }
249
  }
250
 
251
  updateArpeggio(handId, newRootNote) {
 
254
 
255
  const freq = Tone.Frequency(newRootNote).toFrequency();
256
  padData.synth.frequency.rampTo(freq, 0.15);
257
+ if (padData.harmonySynth) padData.harmonySynth.frequency.rampTo(freq * 1.498, 0.15);
258
+ if (this.subBass) this.subBass.frequency.rampTo(freq / 2, 0.2);
259
+
260
+ // Update continuous synth to track root
261
+ try {
262
+ if (this.middleContinuous) this.middleContinuous.frequency.rampTo(freq * 1.498, 0.2);
263
+ } catch(e) {}
264
+
265
  padData.currentRoot = newRootNote;
266
  }
267
 
 
273
  this.handVolumes.set(handId, clamped);
274
  const db = -30 + clamped * 26;
275
  padData.synth.volume.rampTo(db, 0.1);
276
+ if (padData.harmonySynth) padData.harmonySynth.volume.rampTo(db - 6, 0.1);
 
 
277
  }
278
 
279
  stopArpeggio(handId) {
 
285
  this.subBass.triggerRelease(Tone.now());
286
  }
287
 
288
+ // Release continuous synths
289
+ if (this.padSynths.size <= 1) {
290
+ this._continuousActive = false;
291
+ try { this.middleContinuous.triggerRelease(Tone.now()); } catch(e) {}
292
+ }
293
+
294
  if (!this._pendingDisposals) this._pendingDisposals = new Set();
295
  const synths = [padData.synth, padData.harmonySynth].filter(Boolean);
296
  for (const synth of synths) {
297
  if (!this._pendingDisposals.has(synth)) {
298
  this._pendingDisposals.add(synth);
299
  setTimeout(() => {
300
+ try { synth.dispose(); } catch(e) {}
301
  this._pendingDisposals.delete(synth);
302
  }, 2000);
303
  }
 
308
  }
309
  }
310
 
311
+ // --- SYNTH HAND (hand 0): Comprehensive Distance-from-Palm Control ---
312
+ // Each finger's distance ratio (0=curled to palm, 1=fully extended) CONTINUOUSLY
313
+ // drives dramatic parameter changes. Triggers still fire on transitions.
314
 
315
  updateGesture(handId, gestureData) {
316
  if (!this.isStarted || this._panicMuted) return;
317
 
318
  const { fingerStates, handVelocity, rootNote } = gestureData;
 
319
  const now = performance.now();
320
  const cooldowns = this.fingerCooldowns.get(handId);
321
  if (!cooldowns) return;
322
 
323
  if (!this._prevExtensions[handId]) {
324
+ this._prevExtensions[handId] = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
325
  }
326
  const prevExt = this._prevExtensions[handId];
327
 
328
+ // Use fingerDistances (Euclidean from palm) as primary control, fall back to extensions
329
+ const dist = gestureData.fingerDistances || gestureData.fingerExtensions || {
330
+ thumb: 0.1, index: 0.1, middle: 0.1, ring: 0.1, pinky: 0.1
 
 
331
  };
332
 
333
+ // Smooth all distances to prevent jitter
334
+ const d = {};
335
+ for (const f of ['thumb', 'index', 'middle', 'ring', 'pinky']) {
336
+ d[f] = this._smooth(this._smoothDist, f, dist[f] || 0);
337
+ }
338
+
339
  const rootFreq = Tone.Frequency(rootNote).toFrequency();
340
 
341
+ // ====================================================================
342
+ // THUMB + INDEX: These are the VOLUME CONTROL fingers (pinch distance)
343
+ // Do NOT assign continuous effects here — they'd be forced on whenever
344
+ // volume is up. Only trigger one-shots on curl→extend transitions.
345
+ // ====================================================================
346
+
347
+ // INDEX: Acid Squelch — trigger only (no continuous filter tied to distance)
348
  if (this.fingerSynths.index) {
349
+ if (prevExt.index < 0.2 && d.index > 0.35 && (now - cooldowns.index) > this.FINGER_COOLDOWN_MS) {
 
 
 
 
 
350
  cooldowns.index = now;
351
  this.fingerSynths.index.triggerAttackRelease(
352
  Tone.Frequency(rootFreq).toNote(), '4n', Tone.now(), 0.8
 
354
  }
355
  }
356
 
357
+ // ====================================================================
358
+ // MIDDLE: FM Chaos — Distance = modulation index (0.5→40)
359
+ // Close = clean tone, extended = insane FM screeching
360
+ // ====================================================================
361
  if (this.fingerSynths.middle) {
362
+ try {
363
+ this.fingerSynths.middle.modulationIndex.value = 0.5 + Math.pow(d.middle, 2) * 40;
364
+ this.fingerSynths.middle.harmonicity.value = 2 + d.middle * 10;
365
+ } catch(e) {}
366
+
367
+ if (prevExt.middle < 0.2 && d.middle > 0.35 && (now - cooldowns.middle) > this.FINGER_COOLDOWN_MS) {
368
  cooldowns.middle = now;
369
+ const zapFreq = rootFreq * Math.pow(2, 3/12);
370
  this.fingerSynths.middle.triggerAttackRelease(
371
  Tone.Frequency(zapFreq).toNote(), '8n', Tone.now(), 0.9
372
  );
373
  }
374
  }
375
+ // Continuous FM pad
376
+ if (this.middleContinuous) {
377
+ try {
378
+ this.middleContinuous.modulationIndex.value = 0.2 + Math.pow(d.middle, 2) * 15;
379
+ this.middleContinuous.volume.rampTo(-28 + d.middle * 14, 0.05);
380
+ } catch(e) {}
381
+ }
382
 
383
+ // ====================================================================
384
+ // RING: Shimmer/Delay — Distance = delay feedback + time warping
385
+ // Close = dry/tight, extended = infinite echoing shimmer
386
+ // ====================================================================
387
  if (this.fingerSynths.ring) {
388
+ if (prevExt.ring < 0.2 && d.ring > 0.35 && (now - cooldowns.ring) > this.FINGER_COOLDOWN_MS) {
 
389
  cooldowns.ring = now;
390
+ const chimeFreq = rootFreq * Math.pow(2, 7/12);
391
  this.fingerSynths.ring.triggerAttackRelease(chimeFreq, '4n', Tone.now(), 0.8);
392
  }
393
  }
394
+ if (this.delay) {
395
+ this.delay.feedback.value = 0.1 + d.ring * 0.65;
396
+ this.delay.wet.value = 0.05 + d.ring * 0.5;
397
+ // Modulate delay time subtly for chorus-like shimmer
398
+ try {
399
+ const delayMod = 0.15 + d.ring * 0.15; // 8th note dotted range
400
+ this.delay.delayTime.value = delayMod;
401
+ } catch(e) {}
402
+ }
403
 
404
+ // ====================================================================
405
+ // PINKY: Sub Drop + Bitcrusher — Distance = crush bits + reverb intensity
406
+ // Close = clean sub, extended = destroyed/crushed + massive reverb
407
+ // ====================================================================
408
  if (this.fingerSynths.pinky) {
409
+ if (prevExt.pinky < 0.2 && d.pinky > 0.35 && (now - cooldowns.pinky) > this.FINGER_COOLDOWN_MS) {
 
410
  cooldowns.pinky = now;
411
  this.fingerSynths.pinky.triggerAttackRelease('C1', '4n', Tone.now(), 0.9);
412
  }
413
  }
414
+ if (this.bitcrusher) {
415
+ // 16 bits (clean) → 2 bits (destroyed)
416
+ this.bitcrusher.bits.value = Math.max(2, Math.round(16 - d.pinky * 14));
417
+ }
418
+ if (this.reverb) {
419
+ this.reverb.wet.value = 0.15 + d.pinky * 0.55;
420
+ }
421
 
422
+ // ====================================================================
423
+ // COMBINED: Hand spread → chorus + detuning + wobble speed
424
+ // ====================================================================
425
+ const spread = gestureData.handSpread || 0;
426
+ if (this.chorus) {
427
+ this.chorus.depth = 0.3 + spread * 0.7;
428
+ this.chorus.frequency.value = 1 + spread * 4;
429
+ }
430
+ if (this.wobbleLFO) {
431
+ this.wobbleLFO.frequency.value = 0.2 + spread * 8;
432
+ }
433
+ this.padSynths.forEach(padData => {
434
+ padData.synth.detune.rampTo(spread * 50, 0.1);
435
+ if (padData.harmonySynth) padData.harmonySynth.detune.rampTo(-spread * 30, 0.1);
436
+ });
437
 
438
  // Store for next frame
439
+ prevExt.thumb = d.thumb;
440
+ prevExt.index = d.index;
441
+ prevExt.middle = d.middle;
442
+ prevExt.ring = d.ring;
443
+ prevExt.pinky = d.pinky;
444
 
445
  // Percussive hits on sharp hand movement
446
  if ((now - this.percCooldown) > this.PERC_COOLDOWN_MS) {
447
  const vx = handVelocity?.x || 0;
448
  const vy = handVelocity?.y || 0;
449
  const magnitude = Math.sqrt(vx * vx + vy * vy);
 
450
  if (magnitude > this.HAND_VELOCITY_THRESHOLD) {
451
  this.percCooldown = now;
452
  if (vy > this.HAND_VELOCITY_THRESHOLD) {
 
458
  }
459
  }
460
 
461
+ // --- DRUM HAND (hand 1): Distance-from-Palm Control ---
 
462
 
463
  updateDrumGesture(gestureData) {
464
  if (!this.isStarted || this._panicMuted) return;
 
467
  const prevExt = this._prevDrumExtensions;
468
  const cooldowns = this._drumFingerCooldowns;
469
 
470
+ const dist = gestureData.fingerDistances || gestureData.fingerExtensions || {
471
+ thumb: 0.1, index: 0.1, middle: 0.1, ring: 0.1, pinky: 0.1
 
 
 
 
472
  };
473
 
474
+ const d = {};
475
+ for (const f of ['thumb', 'index', 'middle', 'ring', 'pinky']) {
476
+ d[f] = this._smooth(this._smoothDrumDist, f, dist[f] || 0);
477
+ }
478
+
479
+ // THUMB: Tempo wobble — distance triggers rhythmic tom hits at varying pitch
480
+ if (this.drumThumbSynth) {
481
+ if (prevExt.thumb < 0.2 && d.thumb > 0.35 && (now - (cooldowns.thumb || 0)) > 200) {
482
+ cooldowns.thumb = now;
483
+ const pitch = 60 + d.thumb * 200; // Hz
484
+ this.drumThumbSynth.triggerAttackRelease(pitch, '16n', Tone.now(), 0.7);
485
+ }
486
+ }
487
+
488
+ // INDEX: Kick — distance controls pitch decay depth
489
  if (this.drumFingerSynths.index) {
490
+ try {
491
+ this.drumFingerSynths.index.pitchDecay = 0.02 + d.index * 0.2;
492
+ this.drumFingerSynths.index.octaves = 4 + d.index * 6;
493
+ } catch(e) {}
494
+ if (prevExt.index < 0.2 && d.index > 0.35 && (now - cooldowns.index) > this.FINGER_COOLDOWN_MS) {
495
  cooldowns.index = now;
496
  this.drumFingerSynths.index.triggerAttackRelease('C1', '8n', Tone.now(), 0.9);
497
  }
498
  }
499
 
500
+ // MIDDLE: Riser — distance = filter sweep position
501
  if (this._riserFilter) {
502
+ this._riserFilter.baseFrequency = 100 + d.middle * 4000;
503
+ this._riserFilter.octaves = 2 + d.middle * 6;
504
  }
505
  if (this.drumFingerSynths.middle) {
506
+ if (prevExt.middle < 0.2 && d.middle > 0.35 && (now - cooldowns.middle) > 200) {
507
  cooldowns.middle = now;
508
  this.drumFingerSynths.middle.triggerAttackRelease('4n', Tone.now(), 0.7);
509
  }
510
  }
511
 
512
+ // RING: Stutter — distance controls burst count and speed
513
  if (this.drumFingerSynths.ring) {
514
+ if (prevExt.ring < 0.2 && d.ring > 0.35 && (now - cooldowns.ring) > 250) {
515
  cooldowns.ring = now;
516
+ const burstCount = Math.round(2 + d.ring * 8); // 2→10 hits
517
+ const burstSpeed = Math.max(15, 60 - d.ring * 40); // ms between hits
518
+ for (let j = 0; j < burstCount; j++) {
519
  setTimeout(() => {
520
  try { this.drumFingerSynths.ring.triggerAttackRelease('64n', Tone.now(), 0.6); } catch(e) {}
521
+ }, j * burstSpeed);
522
  }
523
  }
524
  }
525
 
526
+ // PINKY: Reverb Crash — distance = reverb size + wet
527
  if (this._crashReverb) {
528
+ this._crashReverb.wet.value = 0.3 + d.pinky * 0.7;
529
  }
530
  if (this.drumFingerSynths.pinky) {
531
+ if (prevExt.pinky < 0.2 && d.pinky > 0.35 && (now - cooldowns.pinky) > 300) {
532
  cooldowns.pinky = now;
533
  this.drumFingerSynths.pinky.triggerAttackRelease('8n', Tone.now(), 0.8);
534
  }
535
  }
536
 
537
+ // CONTINUOUS: chorus from average drum distance
538
+ const avgDist = (d.index + d.middle + d.ring + d.pinky) / 4;
539
  if (this.chorus) {
540
+ this.chorus.depth = 0.3 + avgDist * 0.7;
541
  }
542
 
543
+ prevExt.thumb = d.thumb;
544
+ prevExt.index = d.index;
545
+ prevExt.middle = d.middle;
546
+ prevExt.ring = d.ring;
547
+ prevExt.pinky = d.pinky;
548
  }
549
 
550
+ // --- Finger Expression (legacy + enhanced) ---
551
 
552
  updateFingerExpression(params) {
553
  if (!this.isStarted) return;
554
 
555
+ const { fingerDistances, middleFinger, ringFinger, pinkyFinger, handSpread } = params;
 
 
 
 
 
 
 
 
 
 
556
 
557
+ // Hand spread wobble LFO speed + pad detune
558
  if (this.wobbleLFO) {
559
  this.wobbleLFO.frequency.value = 0.2 + handSpread * 6;
560
  }
 
 
561
  this.padSynths.forEach(padData => {
562
  padData.synth.detune.rampTo(handSpread * 40, 0.1);
563
+ if (padData.harmonySynth) padData.harmonySynth.detune.rampTo(-handSpread * 25, 0.1);
 
 
564
  });
565
  }
566
 
 
596
  const cutoff = 16000 * Math.pow(0.075, value);
597
  this.filter.frequency.rampTo(cutoff, 0.2);
598
  }
599
+ if (this.reverb) this.reverb.wet.value = 0.3 + value * 0.3;
600
+ if (this.delay) this.delay.wet.value = 0.15 + value * 0.2;
601
+ if (this.chorus) this.chorus.depth = 0.4 + value * 0.4;
 
 
 
 
 
 
602
  }
603
 
604
+ // --- PANIC ---
605
+
606
  panic() {
607
  this._panicMuted = true;
608
  setTimeout(() => { this._panicMuted = false; }, 1000);
609
 
 
610
  this.padSynths.forEach((padData) => {
611
  try { padData.synth.triggerRelease(Tone.now()); } catch(e) {}
612
  try { if (padData.harmonySynth) padData.harmonySynth.triggerRelease(Tone.now()); } catch(e) {}
 
619
  this.handVolumes.clear();
620
  this.fingerCooldowns.clear();
621
 
622
+ if (this.subBass) try { this.subBass.triggerRelease(Tone.now()); } catch(e) {}
623
+
624
+ // Release continuous synths
625
+ try { this.middleContinuous.triggerRelease(Tone.now()); } catch(e) {}
626
+ this._continuousActive = false;
627
 
 
628
  if (this.fingerSynths) {
629
  for (const finger of ['index', 'middle', 'ring', 'pinky']) {
630
  try { this.fingerSynths[finger].triggerRelease(Tone.now()); } catch(e) {}
631
  }
632
  }
 
 
633
  if (this.drumFingerSynths) {
634
  for (const finger of ['index', 'middle', 'ring', 'pinky']) {
635
  try { this.drumFingerSynths[finger].triggerRelease(Tone.now()); } catch(e) {}
636
  }
637
  }
638
 
 
639
  if (this.wobbleLFO) {
640
  this.wobbleLFO.stop();
641
  setTimeout(() => { try { this.wobbleLFO.start(); } catch(e) {} }, 1000);
642
  }
643
 
644
+ if (this.filter) this.filter.frequency.value = 16000;
 
 
 
 
 
645
  if (this.reverb) this.reverb.wet.value = 0.35;
646
  if (this.delay) this.delay.wet.value = 0.2;
647
  if (this.chorus) this.chorus.depth = 0.6;
648
+ if (this.bitcrusher) this.bitcrusher.bits.value = 16;
649
 
650
+ this._prevDrumExtensions = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
651
+ this._drumFingerCooldowns = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
652
+ this._smoothDist = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
653
+ this._smoothDrumDist = { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
654
 
655
  console.log('PANIC — all sound killed, muted for 1 second');
656
  }
game.js CHANGED
@@ -831,20 +831,40 @@ export var Game = /*#__PURE__*/ function() {
831
  _this1.musicManager.updateArpeggio(i, note);
832
  }
833
  _this1.musicManager.updateArpeggioVolume(i, velocity);
834
- // --- Finger Expression Controls ---
835
- // Each finger's curl amount (0=curled, 1=extended) based on tip vs PIP joint Y
836
  var wrist = smoothedLandmarks[0];
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
837
  var middleTip = smoothedLandmarks[12];
838
  var middlePip = smoothedLandmarks[10];
839
  var ringTip = smoothedLandmarks[16];
840
  var ringPip = smoothedLandmarks[14];
841
  var pinkyTip = smoothedLandmarks[20];
842
  var pinkyPip = smoothedLandmarks[18];
843
- // Finger extension: how far tip is above its PIP joint (normalized)
844
- var palmSize = Math.abs(wrist.y - smoothedLandmarks[9].y) || 0.1;
845
  var middleExt = Math.max(0, Math.min(1, (middlePip.y - middleTip.y) / palmSize));
846
  var ringExt = Math.max(0, Math.min(1, (ringPip.y - ringTip.y) / palmSize));
847
  var pinkyExt = Math.max(0, Math.min(1, (pinkyPip.y - pinkyTip.y) / palmSize));
 
848
  // Hand spread: average distance between adjacent fingertips / palm size
849
  var tips = [smoothedLandmarks[4], smoothedLandmarks[8], middleTip, ringTip, pinkyTip];
850
  var spreadSum = 0;
@@ -855,14 +875,15 @@ export var Game = /*#__PURE__*/ function() {
855
  }
856
  var handSpread = Math.max(0, Math.min(1, (spreadSum / 4) / (palmSize * 2)));
857
  _this1.musicManager.updateFingerExpression({
 
858
  middleFinger: middleExt,
859
  ringFinger: ringExt,
860
  pinkyFinger: pinkyExt,
861
  handSpread: handSpread
862
  });
863
  // --- Gesture-Driven Music ---
864
- var indexExt = Math.max(0, Math.min(1, (smoothedLandmarks[6].y - smoothedLandmarks[8].y) / palmSize));
865
  var fingerStates = {
 
866
  index: indexExt > 0.3,
867
  middle: middleExt > 0.3,
868
  ring: ringExt > 0.3,
@@ -871,31 +892,32 @@ export var Game = /*#__PURE__*/ function() {
871
  // Track previous states and compute velocities
872
  if (!_this1._prevFingerStates) _this1._prevFingerStates = {};
873
  if (!_this1._prevHandPos) _this1._prevHandPos = {};
874
- var prevFS = _this1._prevFingerStates[i] || { index: false, middle: false, ring: false, pinky: false };
 
875
  var prevHP = _this1._prevHandPos[i] || { x: hand.anchorPos.x, y: hand.anchorPos.y };
 
876
  var handVelX = hand.anchorPos.x - prevHP.x;
877
  var handVelY = hand.anchorPos.y - prevHP.y;
878
  _this1.musicManager.updateGesture(i, {
879
  fingerStates: fingerStates,
880
  prevFingerStates: prevFS,
 
 
881
  fingerExtensions: {
 
882
  index: indexExt,
883
  middle: middleExt,
884
  ring: ringExt,
885
  pinky: pinkyExt
886
  },
887
- fingerVelocities: {
888
- index: Math.abs(indexExt - (prevFS.index ? 1 : 0)),
889
- middle: Math.abs(middleExt - (prevFS.middle ? 1 : 0)),
890
- ring: Math.abs(ringExt - (prevFS.ring ? 1 : 0)),
891
- pinky: Math.abs(pinkyExt - (prevFS.pinky ? 1 : 0))
892
- },
893
  handVelocity: { x: handVelX, y: handVelY },
 
894
  rootNote: note,
895
  volume: velocity
896
  });
897
  _this1._prevFingerStates[i] = fingerStates;
898
  _this1._prevHandPos[i] = { x: hand.anchorPos.x, y: hand.anchorPos.y };
 
899
  } else {
900
  // If it is a fist, make sure the arpeggio is stopped
901
  _this1.musicManager.stopArpeggio(i);
@@ -908,19 +930,33 @@ export var Game = /*#__PURE__*/ function() {
908
  });
909
  // Per-finger drum sounds via updateDrumGesture
910
  var drumWrist = smoothedLandmarks[0];
911
- var drumPalmSize = Math.abs(drumWrist.y - smoothedLandmarks[9].y) || 0.1;
912
- var drumIndexExt = Math.max(0, Math.min(1, (smoothedLandmarks[6].y - smoothedLandmarks[8].y) / drumPalmSize));
913
- var drumMiddleExt = Math.max(0, Math.min(1, (smoothedLandmarks[10].y - smoothedLandmarks[12].y) / drumPalmSize));
914
- var drumRingExt = Math.max(0, Math.min(1, (smoothedLandmarks[14].y - smoothedLandmarks[16].y) / drumPalmSize));
915
- var drumPinkyExt = Math.max(0, Math.min(1, (smoothedLandmarks[18].y - smoothedLandmarks[20].y) / drumPalmSize));
 
 
 
 
 
 
 
 
 
 
 
 
916
  if (_this1.musicManager.updateDrumGesture) {
917
  _this1.musicManager.updateDrumGesture({
918
  fingerStates: fingerStates,
 
919
  fingerExtensions: {
920
- index: drumIndexExt,
921
- middle: drumMiddleExt,
922
- ring: drumRingExt,
923
- pinky: drumPinkyExt
 
924
  }
925
  });
926
  }
 
831
  _this1.musicManager.updateArpeggio(i, note);
832
  }
833
  _this1.musicManager.updateArpeggioVolume(i, velocity);
834
+ // --- Finger Distance-from-Palm Controls ---
835
+ // Each finger's Euclidean distance from palm center, normalized by palm size
836
  var wrist = smoothedLandmarks[0];
837
+ var palmCenter = smoothedLandmarks[9]; // middle finger MCP = palm center
838
+ var palmDist = Math.sqrt(Math.pow(wrist.x - palmCenter.x, 2) + Math.pow(wrist.y - palmCenter.y, 2)) || 0.05;
839
+ // Fingertip landmarks: thumb=4, index=8, middle=12, ring=16, pinky=20
840
+ var fingerTips = {
841
+ thumb: smoothedLandmarks[4],
842
+ index: smoothedLandmarks[8],
843
+ middle: smoothedLandmarks[12],
844
+ ring: smoothedLandmarks[16],
845
+ pinky: smoothedLandmarks[20]
846
+ };
847
+ var fingerDistances = {};
848
+ for (var fname in fingerTips) {
849
+ var tip = fingerTips[fname];
850
+ var fdx = tip.x - palmCenter.x;
851
+ var fdy = tip.y - palmCenter.y;
852
+ var fdist = Math.sqrt(fdx * fdx + fdy * fdy);
853
+ // Normalize: 0 = touching palm, 1 = fully extended (~ 2x palm size)
854
+ fingerDistances[fname] = Math.max(0, Math.min(1, fdist / (palmDist * 2)));
855
+ }
856
+ // Also keep legacy extension values for compatibility
857
+ var palmSize = palmDist;
858
  var middleTip = smoothedLandmarks[12];
859
  var middlePip = smoothedLandmarks[10];
860
  var ringTip = smoothedLandmarks[16];
861
  var ringPip = smoothedLandmarks[14];
862
  var pinkyTip = smoothedLandmarks[20];
863
  var pinkyPip = smoothedLandmarks[18];
 
 
864
  var middleExt = Math.max(0, Math.min(1, (middlePip.y - middleTip.y) / palmSize));
865
  var ringExt = Math.max(0, Math.min(1, (ringPip.y - ringTip.y) / palmSize));
866
  var pinkyExt = Math.max(0, Math.min(1, (pinkyPip.y - pinkyTip.y) / palmSize));
867
+ var indexExt = Math.max(0, Math.min(1, (smoothedLandmarks[6].y - smoothedLandmarks[8].y) / palmSize));
868
  // Hand spread: average distance between adjacent fingertips / palm size
869
  var tips = [smoothedLandmarks[4], smoothedLandmarks[8], middleTip, ringTip, pinkyTip];
870
  var spreadSum = 0;
 
875
  }
876
  var handSpread = Math.max(0, Math.min(1, (spreadSum / 4) / (palmSize * 2)));
877
  _this1.musicManager.updateFingerExpression({
878
+ fingerDistances: fingerDistances,
879
  middleFinger: middleExt,
880
  ringFinger: ringExt,
881
  pinkyFinger: pinkyExt,
882
  handSpread: handSpread
883
  });
884
  // --- Gesture-Driven Music ---
 
885
  var fingerStates = {
886
+ thumb: fingerDistances.thumb > 0.3,
887
  index: indexExt > 0.3,
888
  middle: middleExt > 0.3,
889
  ring: ringExt > 0.3,
 
892
  // Track previous states and compute velocities
893
  if (!_this1._prevFingerStates) _this1._prevFingerStates = {};
894
  if (!_this1._prevHandPos) _this1._prevHandPos = {};
895
+ if (!_this1._prevFingerDistances) _this1._prevFingerDistances = {};
896
+ var prevFS = _this1._prevFingerStates[i] || { thumb: false, index: false, middle: false, ring: false, pinky: false };
897
  var prevHP = _this1._prevHandPos[i] || { x: hand.anchorPos.x, y: hand.anchorPos.y };
898
+ var prevFD = _this1._prevFingerDistances[i] || { thumb: 0, index: 0, middle: 0, ring: 0, pinky: 0 };
899
  var handVelX = hand.anchorPos.x - prevHP.x;
900
  var handVelY = hand.anchorPos.y - prevHP.y;
901
  _this1.musicManager.updateGesture(i, {
902
  fingerStates: fingerStates,
903
  prevFingerStates: prevFS,
904
+ fingerDistances: fingerDistances,
905
+ prevFingerDistances: prevFD,
906
  fingerExtensions: {
907
+ thumb: fingerDistances.thumb,
908
  index: indexExt,
909
  middle: middleExt,
910
  ring: ringExt,
911
  pinky: pinkyExt
912
  },
 
 
 
 
 
 
913
  handVelocity: { x: handVelX, y: handVelY },
914
+ handSpread: handSpread,
915
  rootNote: note,
916
  volume: velocity
917
  });
918
  _this1._prevFingerStates[i] = fingerStates;
919
  _this1._prevHandPos[i] = { x: hand.anchorPos.x, y: hand.anchorPos.y };
920
+ _this1._prevFingerDistances[i] = Object.assign({}, fingerDistances);
921
  } else {
922
  // If it is a fist, make sure the arpeggio is stopped
923
  _this1.musicManager.stopArpeggio(i);
 
930
  });
931
  // Per-finger drum sounds via updateDrumGesture
932
  var drumWrist = smoothedLandmarks[0];
933
+ var drumPalmCenter = smoothedLandmarks[9];
934
+ var drumPalmDist = Math.sqrt(Math.pow(drumWrist.x - drumPalmCenter.x, 2) + Math.pow(drumWrist.y - drumPalmCenter.y, 2)) || 0.05;
935
+ var drumFingerTips = {
936
+ thumb: smoothedLandmarks[4],
937
+ index: smoothedLandmarks[8],
938
+ middle: smoothedLandmarks[12],
939
+ ring: smoothedLandmarks[16],
940
+ pinky: smoothedLandmarks[20]
941
+ };
942
+ var drumFingerDistances = {};
943
+ for (var dfname in drumFingerTips) {
944
+ var dtip = drumFingerTips[dfname];
945
+ var dfdx = dtip.x - drumPalmCenter.x;
946
+ var dfdy = dtip.y - drumPalmCenter.y;
947
+ var dfdist = Math.sqrt(dfdx * dfdx + dfdy * dfdy);
948
+ drumFingerDistances[dfname] = Math.max(0, Math.min(1, dfdist / (drumPalmDist * 2)));
949
+ }
950
  if (_this1.musicManager.updateDrumGesture) {
951
  _this1.musicManager.updateDrumGesture({
952
  fingerStates: fingerStates,
953
+ fingerDistances: drumFingerDistances,
954
  fingerExtensions: {
955
+ thumb: drumFingerDistances.thumb,
956
+ index: drumFingerDistances.index,
957
+ middle: drumFingerDistances.middle,
958
+ ring: drumFingerDistances.ring,
959
+ pinky: drumFingerDistances.pinky
960
  }
961
  });
962
  }