[sethbern]

This improves the LLM peer prompting after running it in a large classroom. It can now see the answer key and the author’s per-option feedback, which it uses for steering only and never quotes back. The main fixes:

• It confirms students who already picked the correct answer instead of interrogating them
• It has to say something of its own in each message, not just ask another question
• It won’t re-ask the same question in different words

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[bnmnetp]

Thanks for this. Giving the model the question author's feedback is a good idea. My main concern is that several of the new rules contradict rules that are still in the prompt, so I'm not sure how the model will behave. All line references are in bases/rsptx/assignment_server_api/routers/peer.py.

Prompt contradictions

1. An older override beats the new behaviour (~L2292). The existing rule "if they repeat the same answer or sound confident, tell them to vote again — this overrides everything else" wins over the new rules "confidence alone is not a match" and "ask a correct picker once to explain why". A student who writes "im pretty sure its B" still gets sent straight to vote.
2. Answers can leak (~L2288). "If they did not answer your last question... say what you think the answer to it is and move on" conflicts with "never state the correct choice before the student has worked their way to it". The answer to a probing question is often the step that decides the choice.
3. The correct choice's feedback is in every prompt (~L2331). It's included even when the student picked a wrong answer. Author feedback usually explains in full why the answer is right. Combined with "say something of your own in every message", the model is likely to paraphrase it, and "never quote or paraphrase" is the only guard. Could you include the correct choice's feedback only once the student has picked it?
4. Two rules disagree about feedback (~L2277). "Never reference feedback or grading" now sits next to a block labelled "written by the question author". Either the model ignores the new context, or it mentions it ("the question says...") and breaks the peer persona.
5. Analogy mode is affected too (~L2278). The PR describes a fix to generic mode, but common_rules and context_suffix also go into the analogy prompt (analogy_preamble + common_rules + flow_rules + context_suffix). "Point at the exact line, value or word" and CS-worded feedback work against "stay fully in theme vocabulary".
6. It can still keep asking (~L2280). The new "go vote" and "confirm and vote, no question" messages break the unchanged rule that every message starts with a question word or "so". They also conflict with "say something of your own then ask". So the model may keep questioning, which is what this PR is trying to fix.

Code

7. Any letter counts as a pick (~L2333). re.findall(r"[A-Za-z]", selected) treats every letter in selected_answer as a picked choice. Please check matches against the real choice labels.
8. Nested lists truncate feedback (~L1909). The lazy <li>.*?</li> in _LI_RE stops at the first inner </li>, so feedback that contains a list is cut short.
9. Stale comment (~L1935). The comment on _get_mcq_context_async still describes a 4-tuple, but the function now returns 5 values.
10. No tests for the rewritten parse_mchoice_htmlsrc (~L1902). Feedback is matched to answers by position. A test would catch feedback that comes before its answer, or an answer with no feedback, either of which attaches feedback to the wrong letter.

Suggestion

Before merging: remove or reword the old "overrides everything" and "never reference feedback" rules, include the correct choice's feedback only after the student has chosen it, and limit the new rules to generic mode (or check them in analogy mode). Items 7–10 are small fixes.

🤖 Generated with Claude Code

[sethbern]

Thanks for the review! I addressed some of these in the most recent push and added tests for the feedback pairing.