Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/repo-assist.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,14 @@ If a selected task is not applicable to the current repository state, Repo Assis
| Task 9: Testing Improvements | Task 5 when no worthwhile coverage or quality gap is identified |
| Task 10: Take the Repository Forward | Task 2 when existing work is complete or blocked and no valuable next step is available |

### When the selected tasks find no work

The three selected tasks are a starting allocation, not a restriction on which tasks Repo Assist may perform. If they and their local fallbacks find no actionable work, Repo Assist follows the `generic_fallback_tasks` plan: testing (9), performance (8), taking the repository forward (10), then documentation/QA (7), engineering (4), and code improvements (5). Initially selected tasks are excluded from the plan, and any task already assessed through a local fallback is skipped.

Each assessment examines a concrete source/test/docs area or existing check; listing issues or test filenames alone is insufficient. The sweep stops at the first verified worthwhile work item and follows the normal validation, memory, and activity-reporting rules. Each task is assessed at most once, preventing fallback cycles. If the bounded sweep finds no suitable work, a justified noop remains valid and useful check results are kept in memory. Explicit command-mode instructions, maintainer holds, PR caps, and other output restrictions still take precedence.

Explicit command mode and scheduled maintenance-only mode receive no generic fallback plan. At the scheduled PR cap, Repo Assist stays with Tasks 6, 1, and 2 and their investigation/triage-only fallbacks.

### Task 1: Issue Labelling

Default weighting: dominates when the label backlog is large.
Expand Down
117 changes: 117 additions & 0 deletions tests/test_repo_assist_generic_fallback.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
import contextlib
import io
import itertools
import json
import os
from pathlib import Path
import tempfile
import textwrap
import unittest
from unittest.mock import patch


WORKFLOW = (Path(__file__).parents[1] / "workflows/repo-assist.md").read_text()
SELECTION = textwrap.dedent(
WORKFLOW.split(" python3 - << 'EOF'\n", 1)[1].split("\n EOF", 1)[0]
)
GENERIC_TASKS = [9, 8, 10, 7, 4, 5]


class GenericFallbackTests(unittest.TestCase):
def select(self, selected=None, issues=None, command="", run_id="123",
repo_assist_prs=0, event="schedule"):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "issues.json").write_text(json.dumps(issues or []))
(root / "prs.json").write_text(json.dumps([
{"title": f"[repo-assist] Fix {index}"}
for index in range(repo_assist_prs)
]))
original_open = open

def local_open(path, *args, **kwargs):
if str(path).startswith("/tmp/gh-aw/"):
path = root / Path(path).name
return original_open(path, *args, **kwargs)

with contextlib.ExitStack() as stack:
stack.enter_context(patch("builtins.open", local_open))
stack.enter_context(patch.dict(
os.environ, GITHUB_RUN_ID=run_id, REPO_ASSIST_COMMAND=command,
GITHUB_EVENT_NAME=event
))
stack.enter_context(contextlib.redirect_stdout(io.StringIO()))
if selected is not None:
random = stack.enter_context(patch("random.Random"))
random.return_value.choices.return_value = selected
exec(compile(SELECTION, "task_selection", "exec"), {})
return json.loads((root / "task_selection.json").read_text())

def test_issue_only_selection_offers_unselected_generic_work(self):
result = self.select([1, 2, 3])
self.assertEqual(result["selected_tasks"], [1, 2, 3])
self.assertEqual(result["generic_fallback_tasks"], GENERIC_TASKS)

def test_selected_testing_performance_and_forward_work_are_not_repeated(self):
result = self.select([9, 8, 10])
self.assertEqual(result["generic_fallback_tasks"], [7, 4, 5])

def test_other_generic_selection_still_falls_back_to_tests_perf_and_forward(self):
result = self.select([7, 4, 5])
self.assertEqual(result["generic_fallback_tasks"], [9, 8, 10])

def test_every_initial_selection_has_a_bounded_nonrepeating_plan(self):
for selected in itertools.combinations(range(1, 11), 3):
with self.subTest(selected=selected):
plan = self.select(list(selected))["generic_fallback_tasks"]
self.assertEqual(plan, [task for task in GENERIC_TASKS if task not in selected])
self.assertTrue(3 <= len(plan) <= 6)
self.assertEqual(len(plan), len(set(plan)))
self.assertFalse(set(plan).intersection(selected))

def test_empty_issue_queue_does_not_disable_proactive_work(self):
result = self.select(issues=[])
self.assertTrue(result["generic_fallback_tasks"])
self.assertEqual(
result["generic_fallback_tasks"],
[task for task in GENERIC_TASKS if task not in result["selected_tasks"]],
)

def test_issue_backlog_does_not_suppress_fallback_plan(self):
result = self.select([1, 2, 3], issues=[{"number": 1, "labels": []}])
self.assertEqual(result["generic_fallback_tasks"], GENERIC_TASKS)

def test_explicit_commands_have_no_generic_fallback_plan(self):
for command in ["Investigate this issue", "Run Task 9", "\nFix the regression\n"]:
with self.subTest(command=command):
result = self.select([1, 2, 3], command=command)
self.assertEqual(result["generic_fallback_tasks"], [])
self.assertEqual(result["selected_tasks"], [1, 2, 3])

def test_blank_command_remains_normal_mode(self):
self.assertEqual(
self.select([1, 2, 3], command=" \n")["generic_fallback_tasks"], GENERIC_TASKS
)

def test_maintenance_only_mode_has_no_generic_fallback_plan(self):
for count in [8, 20]:
with self.subTest(count=count):
result = self.select([9, 8, 10], repo_assist_prs=count)
self.assertTrue(result["maintenance_only"])
self.assertEqual(result["selected_tasks"], [6, 1, 2])
self.assertEqual(result["generic_fallback_tasks"], [])
self.assertEqual(result["new_pr_limit"], 0)

def test_below_capacity_and_manual_runs_still_offer_generic_work(self):
for count, event in [(7, "schedule"), (20, "workflow_dispatch")]:
with self.subTest(count=count, event=event):
result = self.select([1, 2, 3], repo_assist_prs=count, event=event)
self.assertFalse(result["maintenance_only"])
self.assertEqual(result["generic_fallback_tasks"], GENERIC_TASKS)

def test_run_seed_is_still_reproducible(self):
self.assertEqual(self.select(run_id="98765"), self.select(run_id="98765"))


if __name__ == "__main__":
unittest.main()
27 changes: 23 additions & 4 deletions workflows/repo-assist.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,7 @@ steps:
- name: Fetch repo data for task weighting
env:
GH_TOKEN: ${{ github.token }}
REPO_ASSIST_COMMAND: ${{ needs.activation.outputs.text || inputs.command }}
run: |
set -euo pipefail
mkdir -p /tmp/gh-aw
Expand Down Expand Up @@ -338,6 +339,9 @@ steps:
if maintenance_only:
chosen = [6, 1, 2]

command_mode = bool(os.environ.get('REPO_ASSIST_COMMAND', '').strip())
generic_fallback_tasks = [] if command_mode or maintenance_only else [t for t in [9, 8, 10, 7, 4, 5] if t not in chosen]

print('=== Repo Assist Task Selection ===')
print(f'Open issues : {open_issues}')
print(f'Unlabelled issues : {unlabelled}')
Expand All @@ -350,6 +354,8 @@ steps:
print(f' Task {t:2d} ({task_names[t]}): weight {w:6.1f}{tag}')
print()
print(f'Selected tasks for this run: ' + ', '.join(f'Task {c} ({task_names[c]})' for c in chosen))
print('Generic fallback tasks if no actionable work is found: ' + ', '.join(
f'Task {t} ({task_names[t]})' for t in generic_fallback_tasks))

result = {
'open_issues': open_issues, 'unlabelled_issues': unlabelled,
Expand All @@ -359,6 +365,7 @@ steps:
'task_names': task_names,
'weights': {str(k): round(v, 2) for k, v in weights.items()},
'selected_tasks': chosen,
'generic_fallback_tasks': generic_fallback_tasks,
}
with open('/tmp/gh-aw/task_selection.json', 'w') as f:
json.dump(result, f, indent=2)
Expand Down Expand Up @@ -419,7 +426,7 @@ After every change to `notes.json`, run `jq empty /tmp/gh-aw/repo-memory/default

Each run, the deterministic pre-step collects live repo data (open issue count, unlabelled issue count, and open Repo Assist PRs), computes a **weighted probability** for each task, and selects **three tasks** for this run using a seeded random draw. The weights and selected tasks are printed in the workflow logs. You will find the selection in `/tmp/gh-aw/task_selection.json`.

**Read the task selection**: at the start of your run, read `/tmp/gh-aw/task_selection.json` and confirm the three selected tasks in your opening reasoning. Execute **those three tasks** (plus the mandatory Task 11). If a selected task is not applicable to the current repo state, substitute its fallback task rather than doing nothing. Record the substitution in the Task 11 run history entry.
**Read the task selection**: at the start of your run, read `/tmp/gh-aw/task_selection.json` and confirm the three selected tasks in your opening reasoning. Start with **those three tasks**. If a selected task is not applicable to the current repo state, substitute its fallback task. The selection is a starting allocation, not a restriction on which tasks you may perform. If no actionable work emerges, perform the generic fallback sweep below before concluding with `noop`. Do Task 11 after substantive work and record any substitutions in its run history entry.

| Selected task | Not applicable when… | Fallback |
|---|---|---|
Expand All @@ -434,6 +441,18 @@ Each run, the deterministic pre-step collects live repo data (open issue count,
| Task 9 (Testing Improvements) | Test coverage is already comprehensive and no gaps identified | Task 5 |
| Task 10 (Take Repo Forward) | In-progress work from memory is blocked or complete; no valuable next step | Task 2 |

### Generic fallback sweep

Command mode and scheduled maintenance-only mode have no generic fallback plan. In maintenance-only mode, stay with Tasks 6, 1, and 2 and their investigation/triage-only fallbacks.

If the selected tasks and their local fallbacks reveal no actionable work, do not stop merely because no issue is fixable or no PR needs attention. Read `generic_fallback_tasks` from `/tmp/gh-aw/task_selection.json` and assess those tasks even though they were not selected. The order is Task 9 (tests), Task 8 (performance), Task 10 (take the repository forward), then Tasks 7, 4, and 5 (documentation/QA, engineering, and code improvements), excluding initially selected tasks.

- Keep a per-run set of tasks already assessed. Visit each task at most once across both the local fallback table and this sweep; skip previously assessed tasks so cycles such as 1 -> 2 -> 1 and 5 -> 9 -> 5 cannot repeat indefinitely.
- Inspect one concrete, relevant source/test/docs area or existing check for each generic task you assess. Issue titles or the presence of test filenames alone do not establish that no useful work exists. For performance, look for a measurable opportunity; do not invent speculative optimizations.
- Stop the generic sweep at the first verified, worthwhile work item and carry it through the task's normal implementation, validation, memory, and reporting requirements. Do not manufacture low-value changes, add dependencies without approval, or repeat work already covered by a PR or maintainer decision.
- Keep this bounded to the six generic categories, each assessed at most once, within the existing run timeout. If all are exhausted or genuinely blocked, use an honest `noop` with the concrete checks and deferrals, and retain concise latest results in memory's `checks`.
- Command mode remains exclusive: do not start this sweep after completing explicit instructions. Honour all output limits, maintainer holds and any maintenance-only/new-PR restrictions; the sweep is not permission to bypass a PR cap or other safety boundary.

The weighting scheme naturally adapts to repo state:

- When unlabelled issues pile up, Task 1 (labelling) dominates.
Expand All @@ -442,7 +461,7 @@ The weighting scheme naturally adapts to repo state:

**Repeat-run mode**: When invoked via `gh aw run repo-assist --repeat`, runs occur every 5–10 minutes. Each run is independent — do not skip a run. Always check memory to avoid duplicate work across runs.

**Progress Imperative**: Your primary purpose is to make forward progress on the repository. A "no action taken" outcome should be rare and only occur when every open issue has been addressed, all labelling is complete, and there are genuinely no improvements, fixes, or triage actions possible. If your memory flags backlog items, **act on them now** rather than deferring.
**Progress Imperative**: Your primary purpose is to make forward progress on the repository. A "no action taken" outcome should follow concrete assessment of the selected tasks, their local fallbacks, and the bounded generic sweep, not merely an empty issue queue. Act on applicable memory backlog items; respect recorded blockers and maintainer decisions rather than manufacturing work.

Always do Task 11 (Update Monthly Activity Summary Issue) after performing work in non-command mode, including when manually dispatched without a command. Command-mode and no-op runs do not update the issue. In all comments and PR descriptions, identify yourself as "Repo Assist".

Expand Down Expand Up @@ -593,7 +612,7 @@ Maintain a single open issue titled `[repo-assist] Monthly Activity {YYYY}-{MM}`
- PRs that should be closed (stale, superseded, etc.)
- Any strategic suggestions (goals, priorities)
Use repo memory and the activity log to compile this list. Include direct links for every item. Keep entries to one line each.
5. Do not update the activity issue if nothing was done in the current run. However, if you conclude "nothing to do", first verify this by checking: (a) Are there any open issues without a Repo Assist comment? (b) Are there issues in your memory flagged for attention? (c) Are there any bugs that could be investigated or fixed? If any of these are true, go back and do that work instead of concluding with no action.
5. Do not update the activity issue for a genuine noop. Before concluding "nothing to do", check unaddressed issues, actionable memory, and fixable bugs, then complete the generic fallback sweep if the selected tasks and local fallbacks found no actionable work. Record concrete checks and blockers rather than claiming the entire repository has no possible improvements.

## Guidelines

Expand All @@ -608,4 +627,4 @@ Maintain a single open issue titled `[repo-assist] Monthly Activity {YYYY}-{MM}`
- **Systematic**: use the backlog cursor to process oldest issues first over successive runs. Do not stop early.
- **Release preparation**: use your judgement on each run to assess whether a release is warranted (significant unreleased changes, changelog out of date). If so, create a draft release PR on your own initiative — there is no dedicated task for this.
- **Quality over quantity**: noise erodes trust. Do nothing rather than add low-value output.
- **Bias toward action**: While avoiding spam, actively seek ways to contribute value within the three selected tasks. A "no action" run should be genuinely exceptional.
- **Bias toward action**: While avoiding spam, start with the three selected tasks and use the bounded generic fallback sweep when they reveal no actionable work. A "no action" run must be backed by concrete checks, not merely the initial task selection.
Loading