Native audio & microphone

System and microphone sources#

The helper uses NAudio WASAPI loopback capture for the default Windows playback device and WASAPI capture for the default microphone. It follows those default devices at capture startup; there is no per-application audio selector in the current interface.

Enable system audio and microphone independently. If a device cannot be opened, the helper reports the error to stderr and that source contributes silence. An audio track being present in the MP4 does not prove a particular device was audible. Verify a short clip after changing Windows devices.

Separate streams and PCM layout#

The helper converts source audio to 48 kHz stereo floating-point samples. Mono sources are duplicated; other layouts are converted before mixing. With --separate, each ten-millisecond chunk carries 480 frames × 6 channels × 2 bytes = 5760 bytes of signed 16-bit PCM: combined left/right, system left/right, microphone left/right. Samples are clamped independently.

FFmpeg receives s16le at 48000 Hz, six channels with a 5.1 transport layout. asplit and pan select the three stereo pairs; this transport is not surround playback. Output is stereo AAC at 192 kbit/s per enabled stream. A default combined stream supports normal players; system and microphone stems follow for enabled sources. Their handler names are HydroClips mix, HydroClips system and HydroClips microphone.

Game-only recording has combined + system; microphone-only has combined + microphone; both enabled have three streams. If both sources are disabled, no audio stream is written. Manual recordings and instant replays preserve all those streams.

Microphone modes#

The voice activity gate checks amplitude rather than recognizing speech. Its threshold is approximately 0.02 and its hold is about 200 milliseconds. This is not noise cancellation. Background noise can open the gate; a quiet microphone can fail to open it.

The gate affects only the microphone. System playback continues when enabled. Use the audio user guide to choose a mode and test the resulting clip.

Diagnose silent or distorted audio#

1. Check the Windows default playback and recording devices.
2. Record with system audio enabled and microphone disabled.
3. Test the microphone separately with Always on.
4. Then enable push to talk or voice activity and compare.
5. Inspect native.log for a device-open failure.

If two loud sources clip when mixed, reduce their source volume. Changing audio settings restarts capture, so save important replay history before making a capture-affecting change.

Implementation reference: native/Program.cs: AudioCapture; src/native/recorder.ts: startCapture · native 2.0.7.

Preview and remix audio#

The editor inspects the file’s actual streams rather than trusting stored clip metadata. It selects labelled source tracks, or the first combined track in an older file. Each selected audio track is extracted locally into an M4A cache using stream copy, with an AAC fallback when necessary. Only registered clip IDs and valid track indices are accepted; a session token protects HTTP range playback. The private cache keeps up to eight generated leaf files.

A muted video supplies the playback clock. HTML audio elements feed Web Audio gain nodes with 0–200% controls; playback, seeking, buffering and pause events keep the source clocks aligned. Selection end pauses all preview media. Export remixes the retained stems into the default AAC track and copies the original stems into the new MP4. Reopening an edit restores its levels, and subsequent edits can restore a previously muted source.

Microphone voice activity and push-to-talk gating happen at capture time and affect both the combined track and microphone stem. They cannot be undone afterwards. Older combined audio cannot be separated.

Implementation reference: src/native/audio-tracks.ts; src/native/clip-editor.ts; ClipEditor.tsx · native 2.0.7.