B.W.A Pose-Gate // Reborn Core Concept

Type: Skeletal-to-Synth/FJ/MIDI Gateway (RC1)
Purpose: Reclaiming physical interactions inside the Crumbforest waldmitte nodes.
Interface: RaySharp/JUAN CCTV Core 🔌 AI Inference Engine 🎛️ MIDI/OSC Synthesizer

🎹 The Concept: The Human as a Synthesizer

Rather than treating local surveillance security cameras as passive, dry monitoring feeds, B.W.A Pose-Gate turns them into expressive, live-performance physical controllers.

By processing our camera feeds on a highly-optimized local CPU/GPU loop, we map the skeleton to a visual, virtual MIDI instrument:

    [ Zosi CAM 4 Camera ]
             |
             | (HTTP Snapshots over Local LAN)
             v
   [ ROI Workspace Crop ] --------> Isolates sitting area
             |
             v
    [ PoseLandmarker AI ] --------> Classifies 33 skeletal landmarks
             |
             +-----> WRISTS  (Height mapped relative to shoulders)  -----> MIDI CC Control (0 - 127)
             |
             +-----> COORDINATORS (Waving, Vector stretch, X/Y)       -----> VJ OSC Signal Coordinates
             |
             v
 [ CrumbVJ and MIDI Outputs ] ----> Resolume / TouchDesigner / Hardware Audio Synths

🎛️ MIDI Controller Mapping Spec

Our modular model (pose_tracker.py) executes automatic translation of skeletal extensions into standard MIDI CC parameter blocks:

Joint Vertex Physical Interaction MIDI Output Spec Purpose / Destination
Right Wrist Y Hand raised/lowered. MIDI CC Channel 1 (0-127) Synth Filter Sweep (Cutoff) / VJ Video Opacity.
Left Wrist Y Hand raised/lowered. MIDI CC Channel 2 (0-127) LFO Speed / VJ Slide effect rate.
Right Wrist X Arm translation (Left/Right). MIDI CC Channel 3 (0-127) Stereo Pan / Multi-camera VJ Mix crossfader.
Shoulder Stretch Hunching or leaning backward. MIDI CC Channel 4 (0-127) Reverb decay / VJ Feedback loop distortion.

🎨 Visual Performance integration (CrumbVJ)

B.W.A Pose-Gate is designed to feed clean frame buffers directly to VJ servers.

Why this is awesome for VJ sets:

  • Interactive Media Loops: The VJ set reacts organically to your body position at the desk. Raising your hand literally lifts the video opacity, speeds up frame feedback loops, or triggers active clip changes on beat.
  • Low Latency Local Loop: Since JAWS feeds standard HTTP JPEGs and Python processes drawing overlays on the local AMD GPU, frame latency ranges under ~40-60ms. Perfect for live, organic, physical VJing.

Autonomous edge execution, waldmitte core-node automation.