Skip to content

Add optional Windows ROCm backend for DiffusionGemma - #1

Open
Yasei-no-otoko wants to merge 1 commit into
githubnext:mainfrom
Yasei-no-otoko:feat/windows-rocm-backend
Open

Yasei-no-otoko wants to merge 1 commit into
githubnext:mainfrom
Yasei-no-otoko:feat/windows-rocm-backend

Conversation

@Yasei-no-otoko

Copy link
Copy Markdown

Add an optional Windows ROCm backend for the original DiffusionGemma checkpoint, validated on Radeon 8060S (gfx1151). LocalJev continues to use its OpenAI-compatible upstream interface: users can connect an existing ROCm server through the normal environment variables, or choose the bundled PyTorch/Transformers adapter. The oMLX defaults and decision/evaluation logic stay unchanged.

The implementation includes:

  • A text Chat Completions adapter with explicit HIP placement, SDPA/eager MoE, dynamic cache, bounded serialized inference, request validation and optional authentication. JSON schemas are prompt instructions; LocalJev retains output validation and corrective retries.
  • A Windows PowerShell launcher with dependency/GPU/port checks, readiness waiting, explicit environment handling and cleanup of its owned backend process tree.
  • An HTTP smoke command covering all three Jev question types, plus a per-request HTTP timeout adjustment so long inference can finish under the Engine's existing upstream timeout policy.

ROCmFPX is not a dependency. It was used only as an external runner for supplementary Q8 validation. This PR includes no ROCmFPX patches, build scripts, models or evaluation dumps. The guide links that historical result separately and does not present its 240/240 coverage as an FP16-adapter result. Python dependencies are optional and deliberately do not install/replace the user's HIP PyTorch wheel.

Validation for this PR:

  • Frozen Bun install; 27 Bun tests; TypeScript type check; 12 CPU-only Python tests: passed.
  • start-rocm.ps1 -CheckOnly passed with Windows PowerShell 5.1 and PowerShell 7 on the gfx1151 host, without loading model weights.
  • Implementation and test files match the previously GPU-validated ROCm change. The guide records two earlier real FP16 /ready/v1/systemone smoke requests on September 20; both validated choice, score and noul outputs.
  • Diff/scope review confirms no changes to the Engine, configuration defaults, evaluator, scoring or lockfile.

OpenAI Codex assisted with the implementation, validation and PR preparation at the contributor's request.

Copilot AI lite review requested due to automatic review settings September 21, 2026 11:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Four moderate issues remain in model loading, rejected-request handling, numeric validation, and request-wide timeout enforcement.

Review effort: Lite
Findings: None

What changed in this PR

Adds an optional Windows ROCm/PyTorch backend for DiffusionGemma, with launch tooling, smoke tests, and documentation.

Changes:

  • Adds ROCm server, optional dependencies, and PowerShell lifecycle management.
  • Adds HTTP smoke tooling and backend tests.
  • Adjusts request timeout handling for long inference.
File Description
test/​test_rocm_server.py Backend validation tests
test/​index.test.ts Timeout regression tests
test/​http-smoke.test.ts HTTP smoke tests
src/​index.ts Inference timeout handling
scripts/​start-rocm.ps1 Windows launcher and cleanup
scripts/​rocm_server.py ROCm-compatible inference backend
scripts/​http-smoke.ts End-to-end smoke command
requirements-rocm.txt Optional Python dependencies
README.md ROCm overview and commands
package.json Bun configuration and scripts
docs/​gfx1151.md Windows ROCm setup guide
.gitignore ROCm runtime exclusions

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants