Repository navigation
fix(queue): stop polling reactions on deleted GitHub comments - #293
Merged
Merged
Conversation
The reaction poll selects every AI GitHub message with feedback = null. When the comment behind a message is deleted on GitHub, the reactions endpoint answers 404, the handler logs "Failed to list reactions ... Not Found" and moves on, and the message stays pending. Prod has 12 such comments on CopilotKit/CopilotKit, and each poll attempt logged all 12 as errors again. On a 404, the handler now checks once per run whether the App can see the repo. If it can, the comment is gone: the message gets the terminal feedback value COMMENT_DELETED, one info line is logged, and the message leaves the polled set. If the repo itself 404s, it is an access problem, not a deletion: nothing is marked and the repo is reported once as an error. Other failures are logged as before and stay pending. Feedback calibration counts only POSITIVE and NEGATIVE, so the new value does not change it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
GITHUB_REACTION_POLLjob logged[GithubReactionPoll] Failed to list reactions for message …: Not Foundfor the same 12 messages on every attempt. This PR stops those messages from being polled again. #292 fixed the timeout in the same job.Root cause
The poll selects every AI GitHub message with
feedback = null. When the comment behind a message is deleted on GitHub, the reactions endpoint returns 404. The handler logged an error, skipped the message, and leftfeedbacknull, so the next run polled it again.Prod evidence (outpost-worker, Oct 28, 12:10 to 12:13 PM PDT): the same 12 comment IDs returned 404, 4 times each, across the job's attempts. All 12 are on
CopilotKit/CopilotKit. A read-onlygh apicall with a user token that has full access to the repo also returns 404 for both the comment and its reactions. The comments are gone. This is not an App permission problem.Fix
When the reactions call returns 404, the handler checks once per run whether the App can see the repo:
feedbackvalueCOMMENT_DELETED. The handler logs one info line and does not poll the message again.feedbackis a free-form string column.feedback-calibrationcounts onlyPOSITIVEandNEGATIVE, and the Discord feedback buttons apply only to Discord tickets, so the new value does not affect either one. The schema comment now documents the new value. This is a comment-only change, so no migration is needed. The job's result data now includesdeleted.Files:
queue/src/handlers/github-reaction-poll.ts, theisGithubNotFoundandisRepoVisiblehelpers inshared/src/integrations/github-client.ts, the tests for both, and the schema comment.Local red-green (real GitHub API, real handler)
Harness: a local Postgres seeded with 256 pending GitHub messages to match prod's count. 12 of them use the real comment IDs that return 404 in prod, and 244 use a live
CopilotKit/CopilotKitcomment. The harness runs the realhandleGithubReactionPolltwice against the real GitHub API with the prod App credentials, under the worker's 30s default timeout. RawGET … 404lines from Octokit are omitted.RED (before #292 and this PR):
GREEN (this branch, on top of #292). A rerun after rebasing onto main with #292 merged gave the same result: 8.1s, then 7.8s, with 0 errors and the same 12 rows marked.
Run 2 polls 244 messages instead of 256 and logs no errors.
Unit tests
New tests fail on the old code and pass on this branch:
COMMENT_DELETEDand logs one info line and no error.isRepoVisibleis called once, and one error is logged.isGithubNotFoundandisRepoVisible(true, false on 404, rethrow on other failures).A non-404 test confirms that transient failures still log and stay pending, and that they do not trigger a repo check.
pnpm turbo run typecheck lint: 21/21 passed. The two changed test files pass (21 tests). Prettier is clean on the changed files.Deploy note
After deploy, the first run marks the 12 known messages and logs 12 info lines. Later runs log no 404 errors for them.