[mudler-agent]

Description

Fix distributed model environment propagation and add optional direct upstream API keys.

A model created through the UI could contain:

backend: cloud-proxy
name: sonnet
env:
  ANTHROPIC_API_KEY: "<token>"
proxy:
  api_key_env: ANTHROPIC_API_KEY
  mode: translate
  provider: anthropic

Local startup applied these overrides, but distributed workers launched the backend without them. The later load RPC carried EnvVars, while cloud-proxy resolved credentials from its process environment. Model loading therefore failed with api_key_env "ANTHROPIC_API_KEY" is unset.

• Carry model environment snapshots through backend installation before worker subprocess startup.
• Preserve model/replica isolation and restart the exact slot when overrides change or are removed.
• Serialize competing installs for the same replica through readiness and operation attachment, including requests with different backend names.
• Add proxy.api_key as an alternative to api_key_env or api_key_file; reject conflicting credential sources.
• Support direct keys in both proxy backends, model-aware failover credential lookup, and a password input in the model configuration form.
• Remove credential-bearing structured config/options logging and document plaintext administrative storage and deployment compatibility.

No linked issue. No deployment or merge is included.

Notes for Reviewers

Direct keys remain plaintext administrative configuration, like model env values. Password masking does not encrypt YAML, admin API responses, backups, or exports. Existing env/file indirection remains available. Updated frontends, workers, and proxy backend binaries are required for the new behavior; see the rollout documentation.

Verification performed:

go test ./core/config ./core/config/meta ./core/backend ./core/services/failover ./backend/go/cloud-proxy ./backend/go/localai-proxy -count=1
go test -race -tags auth ./core/services/worker -count=1
go test -race -tags auth ./core/services/workerctl ./core/services/nodes ./core/services/worker ./pkg/model -count=1 -ginkgo.focus='Install environment wire compatibility|Router install environment|Worker model environment|Model env isolation|coalesces concurrent installs|serializes installs'
cd core/http/react-ui
npm run test:unit
./node_modules/.bin/eslint src/components/ConfigFieldRenderer.jsx tests/unit/configpassword.test.mjs
npm run build

All commands above passed. The broader auth-enabled run passed workerctl, worker, and pkg/model; nodes timed out after 600 seconds, so the full nodes suite is not verified green. Full UI lint reports two hook-rule errors in the unchanged e2e/coverage-fixtures.js; changed-file lint passes. A full native make build and live provider requests were not run.

Regression tests were observed failing before fixes, including missing install environment serialization and concurrent same-slot startup. Independent review covered both changes; the discovered slot concurrency defect was fixed with a real-child regression.

Signed commits

• Commits include maintainer-operated automation sign-off and Assisted-by trailers.
• Documentation updated (docs/content/) for user-facing changes.

[mudler-agent]

Rebased onto master at 2f5ec65 and pushed 2d35bf4. Fixed the stale endpoint command stub signatures that caused the lint, Linux, and Apple typecheck failures; added an interface conformance assertion. All core/pkg test packages compile with auth+debug tags, the full affected endpoint package passes, and the focused distributed race checks pass after rebase. Independent review confirmed a clean patch replay and the bounded fix. The prior SYCL failure was an HTTP/2 error downloading Go modules; the new push starts a fresh build. New CI is queued/running, not yet verified green.