Skip to content

feat: add the credly template release queue and the badge template filter - #5166

Merged
ulemons merged 1 commit into
mainfrom
feat/issue-940-credly-release-foundations
Nov 2, 2026
Merged

ulemons merged 1 commit into
mainfrom
feat/issue-940-credly-release-foundations

Conversation

@ulemons

@ulemons ulemons commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Why

A template that leaves the held queue only gets the badges Credly changes from that moment on; everything before stays out. In prod today that is LFC131 (87k badges), the FinOps certifications, Kubestronaut and the KubeCon speaker badges, about 170k badges waiting on a mapping decision. Their history has to be re-read per template once they are mapped. This PR lays the foundations; the poll walk and the resolve script hook come in the next one.

What

  • Client: optional badgeTemplateId on fetchBadgesPage, sent as filter[badge_template_id] inside the state_updated_at window. Verified by hand on prod: LFC131 lists 250 badges for October against ~12k for the whole org.
  • Table integration."credlyTemplateReleases": one row per template to re-read, with floor, top, the open window cursor (since, until, page), queuedAt and completedAt. Partial index on the pending rows.
  • DAL: queue (re-queuing restarts a template from the new bounds), next pending in queue order, save cursor, complete. Cursor writes and completion carry the queuedAt they read, so a walk that was re-queued underneath gets 0 rows and stops.

Testing

  • Client test for the extra filter.
  • DAL tests (CI): queue order and completion, cursor kept in progress, re-queue restarts and invalidates the stale walker, floor must sit below top.

Refs linuxfoundation/lfx-cdp-insights-ops#940

…lter

Refs linuxfoundation/lfx-cdp-insights-ops#940

Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
@ulemons ulemons self-assigned this Nov 2, 2026
Copilot AI balanced review requested due to automatic review settings November 2, 2026 11:24
@cursor

cursor Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Introduces integration state that will orchestrate large per-template Credly backfills; concurrency via queuedAt is important once the poller lands, but this PR does not enable that path yet.

Overview
Adds groundwork to re-read historical Credly badges per badge template after templates leave the held queue, without pulling the whole org on every page.

Credly client: fetchBadgesPage accepts optional badgeTemplateId, sent as filter[badge_template_id] alongside the existing state_updated_at window (test updated).

Persistence: New integration."credlyTemplateReleases" table (one row per template, floor/top bounds, paginated window cursor, queue/completion timestamps, partial index on pending rows).

DAL: queueCredlyTemplateRelease (upsert restarts a template), nextPendingCredlyTemplateRelease, saveCredlyTemplateReleaseCursor, and completeCredlyTemplateRelease. Cursor updates and completion are keyed by queuedAt so a stale in-flight walk gets 0 rows if the template is re-queued. Follow-up work (poll walk / resolve hook) is explicitly out of scope here.

Reviewed by Cursor Bugbot for commit 536b916. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Transaction-stable now() leaves queuedAt unchanged on re-queue, allowing a stale release to overwrite the restarted cursor or incorrectly complete the new release.

1 open issue
What changed in this PR

Adds the foundations for replaying historical Credly badges after a template is mapped. The polling walk and resolve-script integration remain deferred to a follow-up PR.

Changes:

  • Adds an optional badge-template filter to the Credly API client.
  • Introduces a persistent release queue with cursor tracking and stale-walker guards.
  • Adds tests for filtering, queue ordering, cursor persistence, re-queuing, and date bounds.
File Description
services/​libs/​data-access-layer/​src/​credly/​releases.ts Adds queue, cursor-save, and completion operations.
services/​libs/​data-access-layer/​src/​credly/​releases.test.ts Tests release queue behavior and validation.
services/​libs/​connectors/​src/​connectors/​credly/​apiClient.ts Supports filtering badges by template ID.
services/​libs/​connectors/​src/​connectors/​credly/​apiClient.test.ts Verifies the template filter alongside existing filters.
backend/​src/​database/​migrations/​V1793617305__credly-template-releases.sql Creates the release table and pending-queue index.

Review effort: Balanced

ON CONFLICT ("badgeTemplateId") DO UPDATE
SET floor = EXCLUDED.floor, top = EXCLUDED.top,
since = NULL, until = NULL, page = 1,
"queuedAt" = now(), "completedAt" = NULL, "updatedAt" = now()`,
@ulemons
ulemons merged commit f18ce74 into main Nov 2, 2026
14 checks passed
@ulemons
ulemons deleted the feat/issue-940-credly-release-foundations branch November 2, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants