Skip to content

Replace the audio player's native control with a scrubbable waveform - #54

Merged
davior merged 1 commit into
mainfrom
claude/sweet-newton-gqcfag
Oct 29, 2026
Merged

davior merged 1 commit into
mainfrom
claude/sweet-newton-gqcfag

Conversation

@davior

@davior davior commented Oct 29, 2026

Copy link
Copy Markdown
Owner

What

Audio assets and audio clips now play through a waveform you can click or drag to seek. It has a play/pause button and a 0:20 / 0:40 readout, and it replaces the browser's plain <audio controls>. Video is unchanged.

How

Frontend. There is a new WaveformPlayer, built on wavesurfer.js v7 (BSD-3, no dependencies).

  • The <audio> element stays where it was and only loses its native controls. wavesurfer is given it as media and draws over it.
  • Everything else wired to that element keeps working unchanged: attachPlayer's start-at seek, transcript click-to-seek, the clip editor's "set point here", and the clip's out-point pause.
  • A clip shows its parent's waveform with the part outside its in/out window dimmed.
  • Keyboard: the waveform is a role="slider". Arrow keys seek ±5 s and Space plays or pauses.

Backend. There is a new GET /api/assets/{id}/peaks, which returns {"data": {"duration", "peaks"}}.

  • wavesurfer is never left to build the waveform itself. Given no peaks, it downloads and decodes the whole file in the tab.
  • Instead, ingest/waveform.py decodes once with ffmpeg at 8 kHz mono, streamed so memory stays flat, and max-pools down to 2000 bars.
  • services/waveforms.py caches the result at <key>.peaks.json beside the file. A clip uses its parent's file, so it shares its parent's cache.
  • It is computed on first view, so there is no migration or backfill. Until the peaks arrive, or if they can't be made (no ffmpeg, a missing file), the player scrubs on a flat line.
  • The cache file:
    • is deleted along with its file;
    • is counted with the derived files in storage usage;
    • is never offered as a stray by the cleanup sweep.

Testing

  • pytest -q: 1441 passed, including the new tests/test_peaks.py. It covers peaks, the cache hit, recomputing a corrupt cache, a clip sharing its parent's peaks, not_audio, 404 for another user's asset, a missing file, no ffmpeg, cleanup on delete, and that the cache is not counted as a stray.
  • npm test: 447 passed, including the new WaveformPlayer.test.tsx. npm run lint, format:check and build are clean.
  • Driven in Chromium against a dev server:
    • a click at 75% seeks to 30 s of 40, and a drag to the middle lands on 20 s;
    • Play works;
    • a clip opens at its in point, with its window shaded;
    • the <audio> src is never reset, and wavesurfer makes zero fetches of the media file.
  • A 60-minute file takes about 21 s on first open (almost all of it ffmpeg's decode) and peaks at 37 MB of memory. Every open after that reads the cached file.

Generated by Claude Code

Audio assets and clips now play through a waveform you can click or drag
to seek, with a play/pause button and a time readout, in place of the
browser's plain <audio controls>.

The <audio> element is kept and only loses its native controls.
wavesurfer.js draws over it as `media`, so the start-at seek, transcript
click-to-seek, the clip editor and the out-point pause all keep working
unchanged. A clip shows its parent's waveform with everything outside its
in/out window dimmed.

The peaks come from a new GET /api/assets/{id}/peaks. It decodes once with
ffmpeg at 8 kHz, streamed so memory stays flat, max-pools to 2000 bars,
and caches the result as <key>.peaks.json beside the file. wavesurfer is
never left to build them itself, because that means downloading and
decoding the whole file in the browser. Until the peaks arrive, or if they
can't be made, the player scrubs on a flat line.

The cache is computed lazily, so there is no migration or backfill. It is
deleted with its file, counted with the derived files in storage usage,
and never offered as a stray.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1geD3VLg4aqTvHVuRLA53
@davior
davior marked this pull request as ready for review October 29, 2026 01:26
@davior
davior merged commit 1e5fea2 into main Oct 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants