[aspadiyath]

CodeTailor only used a course's openai token and always called api.openai.com. Courses can now save a umgpt token instead (entered through "Other" on the add-token page), and CodeTailor sends those requests to U-M's gateway (https://api.toolkit.umgpt.umich.edu/v1), the same one async peer instruction now uses (35d10da).

Changes

• openai_token.py: the token lookup returns the key together with its provider. A small LLM_PROVIDERS table gives each provider's base URL and model.
• get_personalized_solution.py: both LLM calls take their client and model from the token. verbosity="low" is only sent to GPT-5 models, because other models reject it.
• coach.py: logs which provider was used. The debug print in parsons_scaffolding was writing the full API key to stdout; it and the log line now show it masked as abcd****wxyz (first and last four characters).

Behavior to be aware of

• If a course has both tokens, CodeTailor uses OpenAI. Peer instruction does the opposite and prefers UMGPT. CodeTailor keeps OpenAI so that courses already running on it don't silently switch models mid-semester. A course that wants UMGPT should remove its OpenAI token.
• On UMGPT, CodeTailor uses gpt-4o-mini, the same default as peer instruction. On OpenAI it stays on gpt-5-nano.

Testing

• New unit tests in test/bases/rsptx/book_server_api/test_openai_token.py cover provider selection, client URL and model, and key masking.
• Tested by hand on localhost with a course that has only a UMGPT key. Get Help on a CSAwesome2 Unit 3 problem returned an AI_personalized fix (generated by the model, passing the question's unit tests), and the log showed provider: umgpt.

🤖 Generated with Claude Code

[bnmnetp]

I think the new files should be in components/rsptx/XXX

maybe we need a new component for llm_helpers. Not sure about the name so if you have better ideas.

[aspadiyath]

Moved all three into components, split by purpose:

• openai_token.py → components/rsptx/ai_helpers/api_token.py. I went with ai_helpers over llm_helpers so it can hold other AI code, and renamed the file since it handles UMGPT tokens too.
• activecode_questions.py and jobe.py → components/rsptx/code_runner/. They aren't AI-specific (they load a question's tests/data files and run them on JOBE), so I put them in their own component.

Happy to rename either.

[bnmnetp]

Thanks for this. Moving the token lookup and JOBE helpers into shared components is a good direction. Below are the issues I found, most important first. Line numbers are approximate.

Bugs (both existed before this PR, but they're in functions it touches)

1. Failed Java test runs count as passing (evaluate_fixed_code.py ~L112). When a file upload to JOBE fails, or JOBE's reply isn't JSON, load_and_run_java_tests returns (False, {...}), and java_unittest_evaluation passes that tuple back as the pass/fail result. A non-empty tuple is truthy, so if unittest_result: treats untested LLM code as passing, and CodeTailor shows students an unverified "fixed" solution.
2. course is never set on the dev path (coach.py ~L520). When uses_dev_api_key(course_name) is true, fetch_course never runs, so getattr(course, "base_course", None) raises UnboundLocalError. The except Exception turns that into a 400 "could not fetch test code", so the dev/test path never reaches the LLM.

Duplication and design

3. A second provider table (components/rsptx/ai_helpers/api_token.py ~L20). LLM_PROVIDERS repeats LLM_BASE_URLS / _llm_provider_async in assignment_server_api/routers/peer.py, and the two now disagree. Peer instruction tries UMGPT first, while CodeTailor tries OpenAI first. Could peer.py use the new shared helper so there's only one copy?
4. The UMGPT model is hardcoded (api_token.py ~L24). It's fixed at gpt-4o-mini and ignores PI_OPENAI_MODEL, which peer instruction uses for the same service. When that model is retired there, every CodeTailor call on UMGPT will fail.
5. verbosity is chosen by model name (get_personalized_solution.py ~L168). Deciding with model.startswith("gpt-5") is a guess. Whether to send it would be better stored with each provider in LLM_PROVIDERS (e.g. an extra_params entry).
6. The Java path still has its own JOBE code (evaluate_fixed_code.py ~L82). load_and_run_java_tests builds its own session, md5 file IDs and base64 encoding instead of using jobe.jobe_session(), jobe.runestone_file_id() and jobe.b64_text_utf8(). Any change to JOBE auth or file IDs would have to be made twice.
7. Dead code was edited instead of deleted (evaluate_fixed_code.py ~L15). _ensure_file_on_jobe, PUSH_PROXY and CHECK_PROXY have no callers, so I'd remove them rather than update them.

Smaller

8. Short keys aren't masked (api_token.py ~L54). masked() shows token[:4] and token[-4:], so an 8-character key prints in full and shorter ones leak completely, in both the logs and repr(). Please mask more for short tokens, or show only a fixed-length prefix.
9. Two ways to load JOBE settings (code_runner/jobe.py ~L13). It imports settings from rsptx.configuration, while the Java path still uses from ..rsproxy import settings. Both resolve to the same object today, but they can drift apart.

The PR description is also out of date: it mentions openai_token.py and test/bases/.../test_openai_token.py, but the diff adds components/rsptx/ai_helpers/api_token.py, with tests under test/components/rsptx/ai_helpers/.

Suggested before merging: #1, #2 and #8, which are all small. Ideally also #3–4, so the two features can't drift apart on providers.

🤖 Generated with Claude Code

[aspadiyath]

Fixed 1, 2 and 8:

• 1: load_and_run_java_tests now returns plain False when a file push fails or JOBE's reply isn't JSON, so those runs count as failing. Added test/bases/rsptx/book_server_api/test_java_jobe_failures.py.
• 2: course is set to None before the dev-key branch, so the dev/test path no longer crashes looking up the question.
• 8: masked() now shows fewer characters for short keys (min(4, len // 6) at each end), so real keys keep their first/last four as a breadcrumb but a short key no longer prints in full.

I'll do 3 and 4 (one shared provider table for PI and CodeTailor, and the UMGPT model setting) as separate PRs after discussing with @sethbern, since they change PI's behavior too.