Skip to content

Import tracks from a saved page, for sites that block GAM (Pixabay) - #50

Merged
davior merged 1 commit into
mainfrom
claude/admiring-planck-ggdijx
Oct 27, 2026
Merged

davior merged 1 commit into
mainfrom
claude/admiring-planck-ggdijx

Conversation

@davior

@davior davior commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Pixabay answers GAM's page fetch with a 403, and its terms rule out automated access. GAM does not disguise itself to get past that. Instead, the person does the part that needs a browser, and GAM never contacts the site:

  1. Save the page in the browser.
  2. Download the wanted tracks with the site's own Download button.
  3. Choose both under Add → "Or use a saved page".

Reading the saved page (page_tracks.extract_saved, no network)

  • Page address: read from the canonical link, og:url, or Chrome's <!-- saved from url=… --> comment, because adapters key on host and path.

  • Pixabay adapter: one rendered row per track. Class names carry build hashes (audioRow--et3ac), so only their stable prefixes are matched. Each row gives:

    • title, artist and track page;
    • duration, genre and tags;
    • the numeric track ID;
    • the listing's /mood/<x>/ (or /genre/) applied to every track.

    On the real saved "mysterious" page, all 20 tracks extract with every field.

  • Other saved pages reuse the existing adapters without the network, so a saved Mixkit page works. A saved incompetech page says to use the link instead, because that page only points to pieces.json.

Matching (page_tracks.match_files, mirrored in utils/matchFiles.ts)

  • By the site's ID in the filename first. Pixabay names downloads artist-title-604094.mp3.
  • Then by title contained in the filename, longest title first ("Tension Riser" before "Riser").
  • One file per track.
  • Shown before upload, so a wrong match is visible while it can still be fixed.

Importing (page_import.import_saved)

POST /api/assets/import-page/saved/preview and POST /api/assets/import-page/saved are multipart and synchronous, like an upload.

  • Every file is stored as music, so it is never transcribed, and is queued for tempo/key/energy analysis.
  • Matched files get the page's details, stamped embedded.
  • Unmatched files are still imported and reported as "not on the page".
  • Checksum duplicates aren't stored again; the page only fills blanks on the existing copy.
  • Non-audio files are refused, and an import takes at most 50 files.

PageTrack.audio_url is now optional, for saved pages with no audio link, and match_id is new. A key property replaces audio_url as the identity in the duplicate helpers. Live imports still require an audio URL.

Testing

  • pytest -q: 1397 passed. The new tests/test_saved_page_import.py (its fixture is a hand-trimmed copy of the Pixabay row structure) covers:
    • extraction and finding the page's address;
    • Mixkit, and incompetech's note;
    • matching by ID, by title, longest title first, and one-to-one;
    • the full import, including that the file is music, isn't transcribed, and is queued for analysis;
    • unmatched and duplicate files;
    • non-audio files and the file limit.
  • npm run format:check, npm run lint, npm test (429 passed, including SavedPageImport.test.tsx) and npm run build all clean.

Generated by Claude Code

Pixabay answers GAM's fetch with 403 and rules out automated access, so
the person does the browser part: save the page, download the tracks,
and hand both to GAM, which never contacts the site.

- page_tracks.extract_saved reads only the uploaded page: its address
  from the canonical link, og:url or Chrome's saved-from comment; a
  Pixabay adapter over the rendered rows (title, artist, track page,
  duration, genre, tags, the listing's mood, and the track ID); and the
  existing adapters without the network for other saved pages.
- match_files pairs downloads to tracks by the ID in the filename, then
  by longest contained title, one to one; mirrored in the browser so
  matches show before upload.
- POST /api/assets/import-page/saved[/preview]: files stored as music
  (no transcription, analysis queued), page details on matched files,
  unmatched ones imported and reported, checksum duplicates not stored.
- Add → "Or use a saved page" in the UI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCL4jm6ogZBBhnjEfBBaXU
@davior
davior marked this pull request as ready for review October 27, 2026 03:47
@davior
davior merged commit 8649a17 into main Oct 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants