feat(editor): add WebXR editing and shared spatial controls - #890
sudhir9297 wants to merge 20 commits into
Conversation
- Add XR terrain brush controls, cursor, and stroke editing - Add undo/redo, snapping, paint scope, and panel scale settings - Expand XR emulator coverage and document the project structure
- Persist wand panel navigation across XR input remounts - Improve emulator targeting and pointer filtering - Advance XR frames without triggering desktop-camera renders
- Add XR settings, roof actions, deletion, and floor controls - Improve XR input capture, wall openings, terrain strokes, and cancellation - Add opening-grid behavior and XR regression checklist
- Add plugin-gated WebXR controls and wand bindings - Register the WebXR plugin and legacy harness compatibility - Include flooring material assets for the editor
- Reuse shared build and editor panel models for WebXR - Remove standalone XR preview routes and wand-panel implementation - Update VR entry points and shared placement controls
- Consolidate Build tab controls and XR plugin integration - Add local-coordinate snapping and wall/roof interaction updates - Move WebXR plugin dependency to a pinned Git revision
|
I hit an error while handling your request (Model unavailable on AI Gateway free tier: Free tier users do not have access to this model. Upgrade to paid credits at https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fai%3Fmodal%3Dtop-up for unrestricted…). Please try again, rephrase, or reach out if it keeps failing. Error id: 5ad4f354-2816-4344-ae32-76cd5fd52314 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Resolve conflicts across editor tooling and node tools: - placement-strategies: keep both getActiveBuildingPose and surface-hit helpers - use-placement-coordinator: adopt shared createShelfStickiness; fix duplicate surfaceWorldPointRef ref - cabinet/tool: take upstream resizable-preview structure; merge effect deps to referenced set - bun.lock: keep plugin-environment/pool/streetscape and both react-three test-renderer + xr Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… on select Drop the 'all' source filter (default to 'pascal') and route the paint panel through armMaterialPaint so selecting a material arms the paint tool. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
| emitter.emit('grid:move', { | ||
| position: [position.x, position.y, position.z], | ||
| localPosition: [local.x, local.y, local.z], | ||
| nativeEvent: { |
There was a problem hiding this comment.
Placement snap uses mismatched building frames
High Severity
XR handle rays convert localPosition with selection.buildingId only, while floor placement now snaps that same point through getActiveBuildingPose(), which falls back to the active level's parent building. When buildingId is missing, the coordinates stay in world space and still get the building yaw applied, so grid snaps land off the visible lattice.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit ac2d6ce. Configure here.
| } | ||
| const hit = new Vector3() | ||
| if (!raycaster.ray.intersectPlane(plane, hit)) return | ||
| if (!intersectSpatialDragPlane(spatialRay ?? raycaster.ray, plane, hit)) return |
There was a problem hiding this comment.
Rotate handle leaks pointer capture
Medium Severity
The group rotate handle calls setPointerCapture and snapshots frozenRest before it knows the gesture can start. Empty participant lists or a missed drag-plane hit return immediately without releasing capture, so later controller or mouse input can stay stuck on a dead rotate session.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit ac2d6ce. Configure here.
| "patchedDependencies": { | ||
| "three@0.185.1": "patches/three@0.185.1.patch", | ||
| "iwer@2.3.0": "patches/iwer@2.3.0.patch" | ||
| } |
There was a problem hiding this comment.
Three.js XR patch never applies
High Severity
The repo pins a compatibility patch to three@0.185.1, but the workspace override forces three to 0.186.0. Bun only applies patches to the exact resolved version, so the WebGPU frame and framebuffer guards never install and immersive rendering can hit the failures those patches were added to prevent.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit ac2d6ce. Configure here.
Recover the roof footprint source helpers silently dropped during the upstream merge and drop the stale three@0.185.1 patch reference now that upstream resolves three@0.186.0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply organize-imports and formatter fixes across editor, viewer, and node tool modules, and declare the missing toolDefaults dependency on the cabinet preview template memo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The grid and ceiling event hooks bail out early when the renderer is presenting an immersive session, so the R3F test harness must supply an xr manager on its mock renderer or the guard dereferences undefined. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Catalog material maps resolve through the assets CDN, so the portable editor fetches them from the hosted origin. Bundling the local .ktx2 copies pushed the release archive past its 70 MB budget; pruning them during staging brings it back to ~51 MB with no functional change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
| }, | ||
| }) | ||
| } | ||
| onPointerDown(event) |
There was a problem hiding this comment.
Failed handle grab keeps XR capture
Medium Severity
InvisibleHandleHitArea captures the XR pointer and emits grid:move before onPointerDown. Resize and rotate sessions are supposed to replace that capture, but useHandleDrag returns immediately when onStart yields no session. The fallback capture then stays armed, so later controller motion drives placement instead of the handle that was grabbed.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 52ef119. Configure here.
| // that context resolves to grid, so it IS the gate. (Previously this also | ||
| // required a ghost in flight, so a merely-armed draft tool showed nothing.) | ||
| const snapPatchVisible = isGridSnapActive() | ||
| const snapPatchVisible = isGridSnapActive() || (immersive && getActiveSnapContext() !== null) |
There was a problem hiding this comment.
XR grid shows when snapping is off
Medium Severity
The placement grid becomes visible in immersive mode whenever getActiveSnapContext() is non-null, even if isGridSnapActive() is false. Angle-only and Off snapping still show the cursor grid patch in VR, so the lattice no longer matches the snap mode.
Reviewed by Cursor Bugbot for commit 52ef119. Configure here.
…camera The WebXR refactor made resolvePointerSupportSurface build the surface-solve ray's second point as origin + normalizedDirection in every case, so t no longer equaled 1 when the pointed surface was the event plane and float drift (2.9999999999999947 vs 3) leaked into placements. Restore the exact worldHit as the second point in the plain perspective case, preserving the pointer-ray and orthographic branches that legitimately have no hit to reuse. Also retarget the slab dependency-tracker test onto createSlabDependencyTracker (the current production path) after levelSlabContextSignatures was removed, and drop the orphaned system.test.ts import of the deleted export. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
There are 6 total unresolved issues (including 5 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 516d76e. Configure here.
| }, | ||
| }) | ||
| } | ||
| onPointerDown(event) |
There was a problem hiding this comment.
XR handle capture leaks on failed start
Medium Severity
InvisibleHandleHitArea always captures the XR pointer before onPointerDown. If the handle session then returns early (failed plane, onStart returning null), that capture is never replaced, so later controller motion still emits grid:move and release still synthesizes pointerup.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 516d76e. Configure here.
1 participant
What does this PR do?
Enter VR from the editor
3106323; local development stays on HTTP, while headsets accessing a LAN address need a secure origin.Headset rendering and navigation
Controller placement and manipulation
Snapping at different scene scales
Shared build and settings controls
Materials and catalog selection
Floor controls in Settings
Older scene loading
Preview links and XR development guidance
How to test
Run
bun installandbun run --cwd apps/editor dev --port 3002, open the main editor, and enable WebXR.Enter VR using a headset on a secure origin or the development emulator, then try God and Human modes.
Place an item, wall opening, and duct terminal, then drag resize and rotate handles.
Compare desktop and spatial settings for a curved wall, roof, stair, and elevator.
Try material selection, painting, and a terrain stroke, then undo.
Add a floor and basement in Settings, switch levels, and remove a non-ground level.
Load an older scene and toggle lightweight preview on a saved-scene URL with other query parameters.
Automated validation passed:
bun run checkandbun run lint(2,048 files).bun run check-types.bun run build.bun teston all 39 changed test files: 155 passed, 0 failed.The production build reports a non-fatal Turbopack filesystem-tracing warning in the MCP SQLite scene store. Headset interaction and the manual steps above still need reviewer verification.
Screenshots / screen recording
Not added yet.
Checklist
bun devbun checkto verify)mainbranchNote
Medium Risk
Touches core editor placement, snapping, handle drags, and viewer rendering paths across desktop and XR; regressions could affect manipulation fidelity even though auth and persistence are unchanged.
Overview
Adds WebXR to the open-source editor via
@webxr/plugin: bootstrap discovery, a feature gate aroundusePascalWebXR, toolbar Enter VR, andimmersive/xrprops onEditorso the viewer can run an immersive session while trimming desktop-only chrome (orbit controls, box select, floating action menus).Build tools are refactored into
build-palette/build-panel-modeland wired to the XR wand throughwebXRWandBindings, including an XR-oriented MEP manifest, roof footprint sources, and tests for select-mode cancellation and tool activation.Spatial input routes controller and hand rays through shared drag-plane and pointer-capture helpers so resize, rotate, wall height, group rotate, and handle moves commit and cancel like desktop; affordance tools preload to avoid first-drag suspense. Placement uses
snapLocalXZInWorldso grid snap stays in model space; pointer support prefers the XR ray over the camera when drawing on the floor; the editor grid transforms with building pose and stays visible under immersive snap.Desktop-adjacent polish: material paint/catalog logic moves into shared panel models; light-preview toggles preserve other URL query params; portable CLI staging drops bundled
.ktx2catalog textures;iwer/ Three.js patches and an XR checklist plus pmndrs MCP config support dev and regression tracking.Reviewed by Cursor Bugbot for commit 516d76e. Bugbot is set up for automated code reviews on this repo. Configure here.