Add optional Windows ROCm backend for DiffusionGemma

.gitignore

@@ -7,3 +7,7 @@ eval/runs/
 .DS_Store
 coverage/
 dist/
+.venv-rocm/
+.runtime/
+__pycache__/
+*.pyc

README.md

@@ -4,7 +4,12 @@ A local, Jev-compatible `POST /v1/systemone` API written in TypeScript for
 [Bun](https://bun.sh/), backed by DiffusionGemma through an OpenAI-compatible
 Chat Completions endpoint.
 
-The defaults target:
+An optional **Windows / AMD ROCm backend**, tested on Radeon 8060S (gfx1151),
+runs the original DiffusionGemma checkpoint in FP16 with PyTorch and
+Transformers. See [the ROCm guide](docs/gfx1151.md) for setup, the PowerShell
+launcher and an end-to-end HTTP smoke check.
+
+The oMLX defaults target:
 
 - inference server: `http://127.0.0.1:8000`
 - model: `diffusiongemma-26B-A4B-it-4bit`
@@ -35,7 +40,7 @@ them for consequential decisions.
 
 ## Run with oMLX
 
-Requires Bun 1.2+ and a running oMLX server.
+Requires Bun 1.4.2+ (for the checked-in lockfile) and a running oMLX server.
 
 ```sh
 bun install
@@ -158,6 +163,7 @@ bun install
 bun test
 bun run typecheck
 bun run smoke       # live call to the configured inference server
+bun run smoke:http  # /ready and all three decision types through LocalJev HTTP
 ```
 
 ## Evaluate different models
@@ -196,7 +202,8 @@ and
 [`lmstudio-ai/lmstudio-bug-tracker#2037`](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/2037).
 The reported MLX backend fails to load `diffusion_gemma`, while the normal llama.cpp
 backend reports an unknown architecture. oMLX already loads and serves your exact
-checkpoint successfully, so it is the better runner for this Mac today.
+checkpoint successfully in the Mac setup. For Windows gfx1151, see the
+[optional ROCm backend](docs/gfx1151.md).
 
 Even after LM Studio adds ordinary generation support, changing runners alone will
 not make the result OpenJev-equivalent. The runner must expose seeded diffusion

docs/gfx1151.md

@@ -0,0 +1,181 @@
+# Windows ROCm / Radeon 8060S (gfx1151)
+
+The optional backend in this guide runs the original
+[`google/diffusiongemma-26B-A4B-it`](https://huggingface.co/google/diffusiongemma-26B-A4B-it)
+checkpoint with ROCm PyTorch and Transformers. LocalJev still generates and
+validates probability JSON; it does not read option probabilities directly
+from logits. The OpenAI-compatible backend implements the text-only subset
+needed by LocalJev.
+
+LocalJev can also connect to an existing OpenAI-compatible ROCm inference
+server through `LOCALJEV_UPSTREAM` and `LOCALJEV_UPSTREAM_MODEL`. Python and
+the adapter below are only needed when using the bundled Transformers
+backend. The existing oMLX defaults remain available.
+
+## Connect an existing ROCm server
+
+Configure the endpoint and the model ID advertised by that server:
+
+```powershell
+$env:LOCALJEV_UPSTREAM = 'http://127.0.0.1:8000'
+$env:LOCALJEV_UPSTREAM_MODEL = 'your-served-model-id'
+bun --no-env-file run src/index.ts
+```
+
+Set `LOCALJEV_UPSTREAM_API_KEY` if the server requires authentication.
+The runner must support the text Chat Completions and model-list endpoints
+used by LocalJev. The instructions below provide a standalone ROCm runner
+for the original DiffusionGemma checkpoint.
+
+## Requirements
+
+- Windows with a working AMD driver and **ROCm-enabled PyTorch** for gfx1151.
+  A CUDA or CPU PyTorch wheel will not work. The backend refuses CPU fallback.
+- A Python environment that already passes a ROCm GPU check (Python 3.13 on
+  the tested host). See [AMD's Windows PyTorch documentation](https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/install/installrad/windows/install-pytorch.html)
+  for driver and wheel installation.
+- Bun **1.4.2 or newer**, matching the upstream lockfile format.
+- About **52 GB of disk space for the model**, plus dependencies, and enough
+  GPU-accessible memory for FP16 weights and working allocations. A 128 GB
+  Strix Halo host is the target; this is not a 4-bit low-memory implementation.
+
+## Set up without changing an existing ROCm environment
+
+Run from the repository root in PowerShell. Set `$RocmPython` to the Python
+executable containing your working ROCm PyTorch installation. Do not rely on
+an unrelated `python` executable earlier on PATH.
+
+```powershell
+$RocmPython = "$env:LOCALAPPDATA\Programs\Python\Python313\python.exe"
+& $RocmPython -c 'import torch; assert torch.version.hip and torch.cuda.is_available(); print(torch.__version__); print(torch.cuda.get_device_properties(0))'
+& $RocmPython -m venv --system-site-packages .venv-rocm
+& .\.venv-rocm\Scripts\python.exe -m pip install -r requirements-rocm.txt
+bun install --frozen-lockfile
+```
+
+The overlay environment reuses the installed ROCm PyTorch while keeping the
+new Transformers version separate. `requirements-rocm.txt` deliberately does
+not install PyTorch. No native compilation is required.
+
+## Start both servers
+
+Check the Python packages, HIP device and available ports without loading
+model weights:
+
+```powershell
+.\scripts\start-rocm.ps1 -CheckOnly
+```
+
+Then start the model and API:
+
+```powershell
+.\scripts\start-rocm.ps1
+```
+
+The first start downloads the official checkpoint through Hugging Face and
+loads it onto the GPU. The launcher waits for the backend before starting
+LocalJev. It listens on loopback, uses one in-flight inference request, and
+stops its own backend when LocalJev exits. Existing services are never stopped
+to free ports. Use another terminal for requests.
+
+For an explicitly downloaded checkpoint, a local model directory also works:
+
+```powershell
+.\scripts\start-rocm.ps1 -Model '.runtime/models/diffusiongemma-26B-A4B-it' -LocalFilesOnly
+```
+
+The served model identifier is the value passed as `-Model`; the launcher
+configures LocalJev to match it. Use `-Python` and `-Bun` for explicit
+executables. The launcher also detects the repository-local Bun executable
+at `.runtime/bun/bun-windows-x64/bun.exe` when present.
+
+The launcher configures its environment directly and does not load `.env`.
+Set optional client authentication in the launching shell:
+
+```powershell
+$env:LOCALJEV_API_KEY = 'your-local-client-key'
+.\scripts\start-rocm.ps1
+```
+
+## Verify the complete HTTP path
+
+```powershell
+Invoke-RestMethod http://127.0.0.1:8080/ready
+bun run smoke:http
+```
+
+`smoke:http` checks readiness, sends a real decision request containing
+`choice`, `score`, and `noul`, and validates the returned probabilities and
+token usage. It prints the upstream model and elapsed time. If client
+authentication is enabled, set `LOCALJEV_API_KEY` in this terminal too.
+For a different API URL, set `LOCALJEV_URL`.
+
+For development checks without loading model weights:
+
+```powershell
+bun test
+bun run typecheck
+& .\.venv-rocm\Scripts\python.exe -m unittest discover -s test -p 'test_rocm_server.py'
+```
+
+## Backend behavior
+
+- FP16, explicit HIP device placement, SDPA attention and eager PyTorch MoE
+  operations are used by default.
+  The startup report includes the actual GPU architecture and package versions.
+- DiffusionGemma uses its own denoising schedule. Ordinary autoregressive
+  `temperature` is not mapped onto that schedule. A dynamic cache avoids the
+  Transformers compilation path on Windows.
+- JSON schemas are included in the prompt. There is no grammar-constrained
+  diffusion decoder; LocalJev retains its validation and corrective retries.
+- Generation is serialized and queue admission is bounded. This is a local
+  development server, not a public multi-user inference service.
+- A different model must be selected explicitly. Generic causal models may
+  be used for diagnostics, but their output does not validate DiffusionGemma.
+
+The original oMLX configuration and evaluation results remain available in the
+[main README](../README.md). Those measurements were made on a Mac and do not
+describe the ROCm backend.
+
+## Verified on September 20, 2026
+
+The official model at revision
+`f7f5b7f5fa82ffc52addd066915886d497f5517b` completed a real
+`/ready` → `/v1/systemone` request on Windows with the following configuration:
+
+| Component | Validated value |
+|---|---|
+| GPU | AMD Radeon 8060S, `gfx1151` |
+| Python | 3.13.9 |
+| PyTorch | `2.15.0a0+rocm10.1.0a20260909` |
+| HIP reported by PyTorch | `7.16.26362` |
+| Transformers | `5.11.0` |
+| Bun | `1.4.2` |
+| Model placement | Entire model on `cuda:0` (PyTorch's HIP device API) |
+| Inference | FP16, SDPA, eager MoE, dynamic cache |
+| Request output ceiling | 512 tokens |
+| Input / output tokens | 522 / 72 |
+| Full HTTP decision latency | 30.80 seconds; 41.47 seconds after a fresh restart |
+
+The smoke request returned `technical` with probability `0.9`, a frustration
+score of `0.9`, and an urgent probability of `0.8`. All three response types
+passed validation without a corrective retry in both runs. Each was the first
+request after loading. This is an integration check,
+not a calibration study or a representative throughput benchmark.
+
+The weights occupy 51,647,701,024 bytes. Loading can temporarily use much more
+system memory than the final GPU allocation. These results validate the
+listed nightly stack; other ROCm versions require their own runtime check.
+
+Development validation passed 27 Bun tests, 12 Python tests, and the TypeScript
+type check. Launcher checks covered Windows PowerShell 5.1 and PowerShell 7,
+including failure, timeout, cancellation and child-process cleanup.
+
+## Separate quantized-runner validation
+
+ROCmFPX was used as an external runner for a separate Q8 evaluation. It is
+not an installation, build or runtime dependency of this ROCm adapter;
+the setup above uses ROCm PyTorch, Transformers and the original checkpoint.
+The [recorded Q8 evaluation](https://github.com/Yasei-no-otoko/localjev/blob/be015c7e41ba8823b5c596db72418ea8a82cdb01/eval/reports/2026-09-20-gfx1151-diffusiongemma-q8-rocmfpx-fa-off/report.md)
+belongs to that different runner and quantization. Its 240/240 valid outputs
+are not a full-suite result for the FP16 adapter described here.

package.json

@@ -4,12 +4,16 @@
   "description": "A Jev-compatible System One API backed by a local OpenAI-compatible model",
   "type": "module",
   "private": true,
+  "engines": {
+    "bun": ">=1.4.2"
+  },
   "scripts": {
     "start": "bun run src/index.ts",
     "dev": "bun --watch run src/index.ts",
     "test": "bun test",
     "typecheck": "tsc --noEmit",
     "smoke": "bun run scripts/live-smoke.ts",
+    "smoke:http": "bun run scripts/http-smoke.ts",
     "eval": "bun run scripts/eval/run.ts",
     "eval:report": "bun run scripts/eval/report.ts"
   },

requirements-rocm.txt

@@ -0,0 +1,5 @@
+# Install into an environment that already exposes a working ROCm PyTorch.
+# Deliberately omit torch: PyPI's default wheel must not replace the HIP build.
+transformers==5.11.0
+accelerate>=1.10,<2
+Pillow>=11,<13