Skip to content

feat(adapters): add CreateOS sandbox provider - #2

Open
BhautikChudasama wants to merge 60 commits into
mainfrom
createos-sandbox
Open

BhautikChudasama wants to merge 60 commits into
mainfrom
createos-sandbox

Conversation

@BhautikChudasama

Copy link
Copy Markdown
Collaborator

Why

CreateOS should be available as a first-class sandbox provider so bot computers can run on managed desktop sandboxes through the same provider-neutral contract as the existing remote providers.

What changed

  • Added a CreateOS sandbox adapter with lifecycle, command, file, browser, screen, and reconnect support.
  • Wired CreateOS into provider selection, docs, examples, and provider canary coverage.
  • Kept noVNC usable when embedded through the isolated screen proxy.
  • Updated mobile Expo package alignment after rebasing on the latest main branch.

Testing

  • pnpm check passed.
  • Targeted Biome checks passed for the CreateOS adapter, its tests, and provider canary coverage.
  • CI previously passed lint, typecheck, unit tests, Postgres journeys, production builds, image validation, and Web E2E on this branch head.

fgomezserna and others added 30 commits September 10, 2026 21:04
* fix(web): retry transient noVNC asset proxy failures

* fix(web): destroy rejected noVNC proxy response before retry

Resume left a slow 5xx body active after upstream was overwritten, so
close/auth handlers only destroyed the newest ClientRequest.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(web): close failed screen responses before retry

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…e222#846)

* fix(mobile): tighten Account copy and show integration logos

Account settings rows and delete account no longer carry explainer
subtitles the confirm flow already covers. Integrations catalog,
featured tiles, and detail reuse catalog logo URLs with an initial
fallback when missing.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(mobile): make Account language a native select

Account language is a read-only value row that opens the shared
action sheet with English and Simplified Chinese only. Russian stays
in catalogs for other surfaces but is no longer offered in this picker.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* style(mobile): satisfy Biome import order on Account

Keep the Account language picker import sorted the way Biome expects
so Lint stays green.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(mobile): make Change password Cancel a plain header action

formSheet was wrapping the custom Cancel control in a filled iOS bar
button. Use a native plain header item with the shared glass background
hidden, and keep a transparent text Pressable for Android.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(mobile): align Maestro screenshots with Android Reply/React/More

Long-press now opens the three-button Alert sheet, so capture React then
emoji and keep gallery publish running when a late assert fails.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(mobile): drop Usage spend explainer on Account

Remove the Model spend subtitle under Usage and the unused
settingsExplanation style. Keep notification switch details as-is.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(mobile): split AccountUiLocale import and harden Maestro sheets

Use a top-level import type for AccountUiLocale. Assert Copy instead of
More on user-message sheets, and capture the reaction after theme frames
so append-only 👍 state cannot leak into dark/light comparisons.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(mobile): match Space default model label in Maestro

Bot settings shows "Space default (…)" when a space model is resolved,
and Maestro treats assertVisible text as a full-string regex, so the
bare "Space default" check failed after Advanced expanded.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
Update stable workspace dependencies and verify E2B sandbox recovery with current SDK errors.
* fix: configure vision for OpenAI-compatible connections

* fix: scope vision capability to selected model

* fix: address vision setting review feedback

* fix: sort openai-compatible import for biome

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: preserve vision for compatible model overrides

* feat: configure per-connection image limits

* fix: preserve connection image limit semantics

* fix: preserve model image settings on updates

* fix: handle image budget and credential recovery

* test: align image budget expectation

* fix: omit visionModelIds when key rotate cannot load prior secret

Keep CodeRabbit recovery for unreadable ciphertext with an explicit
apiKey, but avoid writing a one-model visionModelIds list that would
clobber other enabled models. Legacy DB supportsImages remains the
fallback until plaintext can be merged again.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: keep model image limits opt-in

* refactor: keep screenshot fallback internal

* fix: allow clearing image prompt limit

* feat: configure OpenAI-compatible model settings

* fix: address CodeRabbit thinking level and token budget reviews

Expose xhigh/max reasoning effort options, persist thinkingLevel from
onboarding and mobile model settings, and reject maxTokens above the
configured context window on connect.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: add mobile model setting translations

* Complete OpenAI model settings translations

* fix: restore modelId for resolveModelKey after rebase

Pass modelId into resolveModelKey from resolveConnectedModel and return
vision/token settings. Drop duplicate zh locale keys introduced when
model-setting strings landed beside main's existing labels.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* chore: keep prior workflow blob for fork push

GitHub App tokens cannot update workflow files on the fork head.
Preserve the pre-rebase workflow blob so the conflict resolution can
be force-pushed to alvins82:fix/openai-compatible-vision-setting.
The one-line main delta (if: always() on gallery publish) stays on
origin/main and on cursor/pr-761-rebase-278b.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: sort Onboarding imports and model-vision formatting

Biome organizeImports and formatter after the main rebase.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: add Russian strings for OpenAI-compatible model settings

Cover the new Advanced model-setting chrome ids so mobile i18n
catalog coverage passes for zh and ru.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
elie222#816) (elie222#850)

* fix(sandboxes): enforce configurable computer creation limit per space (elie222#816)

* fix(sandboxes): serialize space computer limit and count legacy containers

Prevent concurrent creates for different bots from racing past
SANDBOX_MAX_COMPUTERS_PER_SPACE by holding a space lock inside the bot
lifecycle lock (bot → space order). Count legacy COMPUTER_IMAGE +
workspaceId computers toward the cap, and match team-screen-style
regex handling for supervisor 429 responses in the Docker adapter.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* style(sandboxes): drop extra blank line in supervisor index

Biome format failed CI on an accidental blank line left beside the
space-container count helper.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* chore(mobile): align Expo 57 patch versions for install --check

Unblock required Typecheck on elie222#850 by bumping expo and sibling expo-*
packages to the ~57.0.x versions expo install --check expects.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(sandboxes): hold space lock across incompatible computer replace

At a configured space cap, serialize remove+create replacements with
fresh admissions so a temporary free slot cannot be claimed by another
bot mid-replace.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* test(sandboxes): assert replace wins and fresh create is rejected

Pin the concurrent replace-vs-create expectations so a swapped outcome
cannot hide a space-cap regression.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: c1borg <c1borg@yahoo.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…ie222#853)

* fix(adapters): normalize OpenAI tool parameters for local servers

Stricter OpenAI-compatible endpoints (e.g. LM Studio) reject tool schemas
that omit type: "object" or properties. Wrap AgentTool.parameters at the
adapter boundary so unions like request_secret and zero-arg tools serialize
with a valid object envelope without inventing fields.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): cast Unsafe tool parameters through unknown

TypeScript rejects a direct TUnsafe → TObject assertion; go through unknown
so the OpenAI wire envelope can still be attached at the adapter boundary.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* fix(adapters): scope ExternalEffect idempotency to run and tool

Provider tool-call ids like call_0 are reused across responses, so treating
them as globally unique caused later mutating tools to replay unrelated
cached results. Key non-approval effects by run, tool, and execution id.

Refs elie222#862

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* style: format effect idempotency key changes

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): harden tool effect idempotency against reused call ids

Include args in the scoped key so same-tool call_0 reuse across turns does
not replay the wrong result, and fall back to legacy bare provider keys only
for the same run, tool, and request (or in-flight effects) during rollout.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): require request match for legacy effect fallback

Incomplete legacy rows with a reused provider id must not attach to a later
mutation with different arguments.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* fix(worker): share one Postgres pool and survive 53300

Prisma, graphile-worker publisher, and graphile-worker runner each opened
their own pool (default max 10), so a worker process could hold ~30
sessions plus a dedicated LISTEN client. When Postgres returned 53300,
graphile's unawaited completeJob became an unhandledRejection and a
container restart opened more backends before the old ones were reaped.

Share the caller's pg.Pool, bound createDb, retry 53300 on connect, treat
connection exhaustion like computer contention during run setup, and do
not exit the worker on that error.

* fix(api): keep a job pool when prisma is overridden

createApp passed an undefined pool into GraphileJobPublisher whenever
prismaOverride was set and WAKEUP_DRIVER was not memory. Open a bounded
createPool for that path, leave DB_POOL_MAX unset in .env.example so the
worker keeps its default of 8, split the PoolClient type import, and only
swallow 53300 on unhandledRejection (still exit on uncaughtException).

Co-authored-by: Khang Huynh <huukhang1999@gmail.com>

* style(db): satisfy biome import order for createPool

Co-authored-by: Khang Huynh <huukhang1999@gmail.com>

* fix(worker): do not exit on startup 53300 exhaustion

Retry jobHost.start on too-many-connections and suppress process.exit(1)
in main().catch for that case so Docker does not crash-loop the worker.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(api): use top-level import type for Pool

Split Pool and PrismaClient out of the value import from @rakazo/db so
they satisfy the top-level import type rule Greptile flagged.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(worker): retry job host start until Postgres has capacity

Replace the finite start retry plus bare main().catch return on 53300 with
an unbounded backoff around jobHost.start so the worker can proceed to
register reconciler and handlers once connections are available, without
crash-looping or staying permanently inactive.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(worker): restart graphile runner after post-start 53300

run() resolving does not mean the runner stays alive; runner.promise can
still reject later. Observe that lifecycle and restart with the same
bounded backoff used around jobHost.start so a swallowed 53300
unhandledRejection cannot leave the process idle with no job runner.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(worker): back off after every runner 53300 and wake sleep on stop

Apply the capacity backoff before each relaunch after runner.promise
rejects with 53300, not only when launchRunner fails, so a runner that
starts then dies under saturation cannot reconnect in a tight loop.
Interrupt the restart delay when stop() runs so SIGTERM is not blocked
for up to 30s waiting on a saturated database.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* feat: rename bot sidebar sections on web and mobile

Users could create bot sections but not fix a typo afterward. Add
botSections.update with the same ownership checks as create, and wire
rename into the Move to menu, section header context menu, and mobile
organize sheet.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: address Greptile review on section rename

Prefer section rename when a deletable space header is also a named
section, resolve inactive-space sections with the correct spaceId for
the update RPC, require HTTP 409 on rename clashes, and add web rename
catalog strings across locales.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: use personal space for bot-menu section rename

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
The title, name field, and Save/Cancel already describe the action.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* fix(api): let a bot answer chat while takeover holds the screen

waiting_takeover blocked new messages as if an ask were pending.
Steer those sends, continue the run, and keep desktop tools gated
until the user releases control.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(api): keep takeover release ahead of a steered continue

Fence continue claims on the observed status and checkpoint so a concurrent
release cannot be overwritten. Stamp a release checkpoint onto a running
steered continue and restore tools or requeue without returning the hold.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…2#880)

* Recover computers stuck in suspending without a database edit

A hung idle stop or Reset left the row in suspending forever.
Team lease acquire and Reset both treated every suspending row as
busy, so operators had no in-product recovery.

Treat suspending claims older than an execution-lease TTL as
abandoned, matching stale booting reclaim, while live suspends
and active runs stay protected.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* Fence suspend reclaim on the first-seen stamp

Idle stop does not refresh updatedAt while checkpointing or
waiting on provider stop. A live suspend could age past the
TTL during the ready wait and be reclaimed while teardown
was still running. Only reclaim stamps that were already
abandoned when first observed, matching booting reclaim.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* chore(mobile): align Expo 57 patch versions for install --check

Unblock required Typecheck by bumping expo and sibling expo-*
packages to the ~57.0.x versions expo install --check expects.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* Fence Reset reclaim and keep recover/update off hung suspends

Reset is the recovery path for a stale suspending row. Recover and
update still refuse so they cannot destroy it without an explicit
Reset. Replacement writes an explicit claim stamp and requires it
on rollback, checkpoint, stop, and error so a later Reset cannot
be overwritten by a hung earlier one.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…222#877)

* fix: allow dot paths through desktop shell guard

* fix: preserve shell quoting and command prefixes

* fix: reject coprocess syntax before sourcing inspection

* fix: reject function syntax and bump expo for typecheck

Bash `function f { . file; }` consumed command position, so the guard
rewrote the following dot builtin as a path and allowed sourcing. Fail
closed on `function` the same way as `coproc`.

Bump expo, expo-image-picker, expo-notifications, and expo-sharing so
mobile `expo install --check` matches the current SDK registry.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: treat then/do/else as reserved only at command position

Ordinary arguments such as `git add then .` were flipping the desktop
shell guard back into command position, leaving the following `.`
unchanged and rejecting a valid repository command.

Co-authored-by: thebestcoder <99405975+snipecoder@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
)

* fix(adapters): pair remote MCP Agent with undici's own fetch

Node 22's built-in fetch is an older undici major than the package Agent.
Handing that Agent in as dispatcher throws before any request is sent.
Drive the guarded lookup with fetch from the same undici, including when
the builtin fetch is injected.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): do not pass the package Agent to custom remote fetch

A wrapper around Node's fetch is not === the builtin, so it still
received the package Agent and threw invalid onRequestStart before
lookup. Attach dispatcher only when calling undici's own fetch.
Custom injections own the transport without that dispatcher.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): pin injected remote fetch to the validated address

Injected fetches cannot use the package Agent, so call them at the
already-checked IP with the original Host instead of resolving again.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): keep TLS hostname when pinning injected remote fetch

Rewriting injected-fetch URLs to the validated IP closed DNS rebinding
but made SNI and certificate checks use that IP. Keep the original
hostname on the request URL and pin Node dns.lookup to the already
validated address for the call, without passing the package Agent.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): pin injected fetch DNS through shared lookup helper

Keep the TLS hostname on injected remote fetches, and pin Node
dns.lookup and dns.promises.lookup to the already-validated address
from a shared helper instead of rewriting the URL to an IP.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): honor address family in pinned DNS all-results

When lookup options request a family with all: true, return only matching
addresses and fail if none exist, matching Node's dns.lookup contract.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): use top-level import type for LookupFunction

Satisfy the repository rule that type-only imports are a separate
import type declaration, not an inline type specifier.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
elie222#839)

* test(adapters): lock bot secrets and connector OAuth out of model view

Add offline conformance coverage that OAuth tokens, refresh tokens,
client secrets, and bot-secret values never appear in model-visible tool
results or review/approval payloads. Close small MCP and Pipedream
redaction gaps so the invariant fails closed.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* style(adapters): split OAuth type imports for lint

Keep top-level import type separate from value imports so Biome and the
repo import rule stay happy.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): keep MCP OAuth redaction fresh and precise

Recompute session secrets from the live OAuth material so refreshed
tokens stay redacted, strip bare Bearer tokens, and only redact
credential-shaped header/env values so ordinary config cannot corrupt
tool output.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): redact cookie and session MCP credentials too

Expand oauthMaterialSecrets header/env matching so Cookie and session
keys are redacted even when short, while ordinary config values still
stay out of the redaction list.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): narrow MCP credential key matching

Match explicit credential carriers (Authorization, Cookie, API tokens,
session ids) instead of any key containing cookie/session, so ordinary
config values are not swept into model-visible redaction.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): fail-closed MCP auth headers without config corruption

Treat Cookie/X-Session/Authorization as always-redacted auth carriers,
including short values, while skipping ordinary config enums such as
production under credential-shaped env keys so global substring
redaction cannot mangle tool output.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): pin MCP redaction material across callTool awaits

Capture session OAuth material before awaiting callTool so concurrent
session eviction cannot drop secrets from model-visible redaction, and
register numeric-only values under explicit credential keys.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* chore(mobile): align Expo 57 patch versions for install --check

Unblock required Typecheck by bumping expo and sibling expo-*
packages to the ~57.0.x versions expo install --check expects.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…22#856)

* feat(ui): match Grokbots look and feel with dark obsidian aesthetic, organic avatars, and refreshed sidebar

* feat(ui): add AvatarStudioPopover with Grok mascot shapes, 10 colors, generate & upload tabs, and update bot settings & sidebar layout

* feat(ui): refine composer action button with circular mic and auto-save bot settings

* feat(ui): overhaul avatar engine and shell with authentic Grokbot aesthetics

* fix(ui): harden geometric avatars and Avatar Studio from elie222#855

Fix operator precedence and hook ordering in BotAvatar, restore system
appearance preference resolution, replace hard-coded shell colors with
semantic tokens, remove the fake Generate tab, reuse ACTIVE_RUN_STATUSES,
and add a Playwright checkpoint for Avatar Studio.

Co-authored-by: Srikanth Doddi <108666666+srikanth-doddi@users.noreply.github.com>

* fix(ui): align CI with geometric avatar and dark token updates

Restore the working-state ring class, sync desktop chrome to the darker
background token, and update tests for header settings plus dual gradients.

Co-authored-by: Srikanth Doddi <108666666+srikanth-doddi@users.noreply.github.com>

* fix(ui): unbreak Web E2E settings and sidebar labels

Rename the conversation header gear so it no longer collides with user
Settings, restore the Integrations label, and use sidebar-accent hover
tokens so bot rows match the integrations row.

Co-authored-by: Srikanth Doddi <108666666+srikanth-doddi@users.noreply.github.com>

* fix(ui): restore Send visibility and drop header settings collision

Remove the conversation-header Settings control that substring-matched
user Settings in Playwright, keep Send always available (with Voice
alongside), and restore semantic inline code colors for light mode.

Co-authored-by: Srikanth Doddi <108666666+srikanth-doddi@users.noreply.github.com>

* fix(ui): repair Web E2E for geometric avatars and shell chrome

Restore settings/computer panel toggle labels, match sidebar row hover to
Integrations, target the working avatar ring in group chats, open Avatar
Studio via bot-settings-trigger, and point the motion fixture at geometric
avatars under reduced motion.

Co-authored-by: Srikanth Doddi <108666666+srikanth-doddi@users.noreply.github.com>

* fix(ui): restore settings/computer panel toggle for Web E2E

Use the shared side-panel header from main so Show computer stays available
from bot settings, and drop sidebar transition-colors that made hover
backgrounds diverge from Integrations mid-animation.

Co-authored-by: Srikanth Doddi <108666666+srikanth-doddi@users.noreply.github.com>

* fix(ui): keep group Speak control reachable with speaker avatars

Flatten the speaker label so the name is not wrapped in a nested span,
and locate Speak this reply from a shared ancestor in the group chats
e2e instead of a single parent hop.

Co-authored-by: Srikanth Doddi <108666666+srikanth-doddi@users.noreply.github.com>

* fix(e2e): wait for group ask cards via threads/get status

Match the choice-card pattern: poll waiting_input on threads/get, then
reload if the ask prompt has not painted yet, instead of requiring a
sidebar status button accessible name.

Co-authored-by: Srikanth Doddi <108666666+srikanth-doddi@users.noreply.github.com>

* fix(ui): harden avatar studio review findings

Serialize bot settings auto-saves, restrict remote avatar image URLs to
data: images, resolve system theme via matchMedia, and restore locale and
a11y gaps called out on the Avatar Studio PR.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* style(web): format bot-panel notification label markup

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(ui): keep auto-save queue on latest state; pack-safe desktop chrome

Route queued bot saves through a ref so older queue entries cannot write
stale empty fields, and keep Electron window chrome on the dark token
literal instead of importing TypeScript-only ui-tokens at runtime.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(ui): share avatar identity color and unstick hover-actions send

Avatar Studio reset and fallback now use DEFAULT_GROK_BOT_COLOR from the
shared palette instead of a duplicated hex. Hover-actions e2e clicks Send
once it is enabled so an in-flight prior send cannot swallow Enter and
leave the long draft in the composer.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(ui): validate avatar hex digits and shape suffixes

Reject non-hex custom colors in resolvePersonaColorDef so persisted
values like #zzzzzz fall back to the identity palette. Normalize
malformed ::shape_ suffixes to shape 0 so BotAvatar and Avatar
Studio stay in sync.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Srikanth Doddi <108666666+srikanth-doddi@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* fix(adapters): bound Pi turns without a default tool cap

Usage records now count billed prompt tokens including cache read and
write, so a cached prompt is no longer stored as ~0. Mutating tool
effects key off run, tool, and args so a restarted run with new model
tool-call ids does not re-apply. Completions use a 2-minute stream
timeout (one retry) and a 4k maxTokens default, overridable through
existing connection settings. When MAX_TOOL_CALLS_PER_TURN is set, the
fuse survives continueRun and finishes a parallel batch before
stopping. Tool results stay clipped to the existing 12k bound.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): keep pause fuse and replay identical-args tools

Keep the optional tool-call budget across ask_user pauses, clip structured
tool results as one aggregate text budget, and key mutating effects so live
identical-args calls still execute while restarts with a new tool-call id
reconcile, including pre-fix rows that used the old call id.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* test(web): wait for send before advancing the delayed focus-card clock

The later-bot setup card is cancelled only after threads.send succeeds.
Fast-forwarding the fake clock immediately after Enter let promptFocus
win the race and post the card while send was still in flight.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(adapters): count effect occurrences before approval keys

Assign live occurrence before choosing an approval vs non-approval
effect key so an approved replay cannot collide with a later
identical-args call. Copy nested pausePending to the parent host so
an optional tool-call fuse survives ask_user inside a subagent.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* style(adapters): fit approval-key assertion on one line

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* test(testkit): drop post-approval duplicate computer_act from the postgres fixture

A later identical computer_act in the same resume is a new occurrence, so the
still-active require_approval rule paused the run again. The fixture still
checks that reconstructed arguments restore the approved request.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…222#863)

Remove unused probed/stored URL parameters from openAiCompatibleConnectReady
(call sites still passed them after readiness stopped requiring a probe), delete
the superseded providerHint helper, and drop the never-referenced portable
browser stop command constant.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* feat: quote a selected text span in message replies

Highlighting text inside a transcript message now offers a floating Quote
action that arms the existing reply composer with the excerpt. The quote
travels beside replyToMessageId through send, persistence, realtime
events, and the reply preview — and the agent sees only the selected
span instead of the whole target message, while reactions keep
full-message context.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: verify reply quote excerpts against the parent message

The excerpt arrives as client input and nothing checked it against the
parent's content, so a modified client could persist a fabricated quote
that renders as a quote and reaches the agent as quotedText.

Flatten both sides before comparing — the selection sees rendered text
while blocks hold markdown source — and drop a mismatch instead of
failing the send, so the reply still lands as a plain reply.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: keep table and list excerpts through quote normalization

A selection across rendered table cells carries no | delimiters, and
alignment rows contribute : — both survived normalization on the parent
side and broke containment, silently dropping legitimate quotes. Strip
them alongside the other markdown punctuation, plus + list markers and
rendered • bullets for the same class of divergence.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: preserve semantic punctuation in quote verification

Blanket character stripping let a fabricated excerpt match by erosion:
"C++ is fast" accepted "C is fast", "key:value" accepted "key value".
Split normalization into two passes — line-level rules remove structural
markdown (table delimiters and alignment rows, list bullets, heading and
blockquote markers, link targets) while semantic punctuation survives on
both sides, so only genuinely matching rendered text verifies.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(api): preserve quotes spanning ordered list items

* fix(api): keep rendered quote numbering significant

* fix(api): retain existing code quote matches

* fix(api): skip ordered-list normalization inside code fences

* fix(api): preserve quote containers when tracking code fences

* fix(api): normalize only source-level ordered list markers
…22#888)

* fix(mobile): stop remounting the computer viewer on every poll

Each computer/screenUrl call seals a new capability URL, and the mobile
viewer is keyed by that URL. Polling re-read it every two seconds, so the
WebView was torn down and reconnected on every poll: the desktop flickered,
the on-screen keyboard closed as soon as it opened, and every release of a
bot desktop looked like a container reboot.

The refresh controller now reads the screen URL only when the status that
shapes it changes, when the held capability nears expiry, on explicit
refreshes after boot/takeover/release, or after the viewer reports the
current URL unusable. The WebView also allows the embedded keyboard bridge
to open the keyboard from a programmatic focus.

* fix(mobile): cache a null screen result until the status changes

A completed screen read is cached even when it returned null, so a stopped or screenless computer no longer re-requests computer/screenUrl on every poll. The viewer still invalidates the cache when a URL stops working.
)

* fix(mobile): adopt UIScene life cycle for iOS 26 SDK builds

* fix(mobile): point the scene manifest at Expo's scene delegate

Use the ExpoAppSceneDelegate that expo@57.0.23 ships instead of a
hand-rolled one, so deep links and universal links flow through Expo's
linking path and the app delegate only provides the RN factory.

* fix(mobile): refuse partially migrated AppDelegate templates

An AppDelegate that already conforms to ExpoReactNativeFactoryProvider
but still contains the legacy window startup would double-start React
Native under the scene life cycle. Fail loudly instead of skipping, and
cover the partial-drift states in tests.

---------

Co-authored-by: absir1949 <5lvt@sina.com>
* fix(voice): let users disconnect a stored voice provider key

Voice settings could connect or replace a provider key but had no way to
remove it. Add voice.disconnect, reuse the existing credential-secret
cleanup path, and show Disconnect next to a connected provider on web and
mobile.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(voice): capture connected settings and freeze provider while busy

Capture the connected Voice settings screen after Disconnect appears so
CI can show the control. Disable provider switching while connect,
disconnect, voice, or test is in flight so a late refresh cannot
overwrite a newer selection.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* test(web): match Voice Connect and Disconnect by exact name

Playwright substring matching treated Disconnect and Connected as
Connect, so the post-disconnect assertion saw two buttons.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(voice): disconnect the stored provider and every matching credential

Disconnect now uses the persisted credential provider rather than the
catalog fallback, and deletes every actor row for that provider so a
leftover duplicate cannot stay listed as connected.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(voice): freeze the voice picker while another operation is pending

Keep the selected-voice control disabled during connect, disconnect, and
test, matching the provider list, so a late setVoice cannot race refresh.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(voice): clear a leftover API key after disconnect

A pasted replacement key stayed in the field after Disconnect. Clear it
the same way Connect does, on web and mobile.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(voice): hold the busy flag while a provider catalog refresh runs

Provider switches started a refresh without pending, so Disconnect could
overlap that load and a late response could restore the old credential.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…e desktop (elie222#891)

* fix(computer): keep the mobile trackpad usable across taps outside the desktop

In trackpad mode, a tap in the free area around the desktop is meant to
click where the pointer is. Two things broke that on iOS:

- The trackpad consumed pointer events, but the touch itself still became
  compatibility mouse events, which reached noVNC and clamped the pointer
  to the nearest desktop edge.
- The synthetic click's mouseup was dispatched on the canvas, whose noVNC
  handler stops propagation, so the mouse-capture overlay noVNC shows on
  mousedown was never released. It then swallowed every following touch
  until a direct touch on the desktop released it.

The trackpad now works on touch events: touches that start on the desktop
are left to noVNC (tap = move the pointer there and click) and only sync
the trackpad cursor; touches that start in the free area are consumed at
the touch level, so they never produce compatibility mouse events, a drag
moves the pointer relatively and a tap clicks where the pointer is. The
synthetic mouseup goes to noVNC's capture overlay when it is shown, which
forwards it to the canvas and releases the capture. A new touch always
starts a fresh gesture, so a lost touch end cannot freeze the trackpad.

* fix(computer): trackpad ignores extra fingers and follows desktop drags

A second finger during a free-area gesture no longer takes it over or clicks when lifted; it is swallowed so it cannot become a compatibility click. A touch that starts on the desktop is tracked until it ends so a drag there keeps the trackpad cursor in step, and the next free-area move continues from the pointer's real position.

* fix(computer): classify every touch of a batched touchstart

A single touchstart can carry several new touches. Each one is now routed on its own: desktop touches are followed, free-area touches drive the trackpad, and the event is consumed whenever it carries a free-area touch so it cannot become compatibility mouse events.
bricelb and others added 29 commits September 17, 2026 17:54
…ainly (elie222#895)

* fix(mobile): give computer boot time to finish and report timeouts plainly

Every mobile RPC is cut off after eight seconds. Booting a computer creates
and starts a container, which routinely takes longer on a small host, so
the app abandoned the request while the server finished the boot and
showed "fetch failed: FetchRequestCanceledException" above a desktop that
then appeared anyway. The boot call now waits up to two minutes, and an
RPC that hits its timeout is reported as "Request timed out" instead of
the native fetch's cancellation detail.

* fix(mobile): abort rpc with an explicit reason shared by every consumer

The controller is aborted with the reason itself (timeout or the caller's cancellation reason), so the fetch, the bounded body read, and nested recovery calls that share the signal all report the same cause. Adds tests for a stalled response body and a caller cancellation.

* fix(mobile): give switching computers the same long timeout as booting

Switching a bot between the Team and a Private computer stops the previous one, which on a small host takes longer than the eight-second RPC default and surfaced as 'Request timed out'. Both call sites now use the lifecycle timeout.

* style(mobile): sort imports
…lie222#896)

* fix(computer): exit promptly on SIGTERM so stops take under a second

The computer image's start script is PID 1 and polled Xvfb in a sleep
loop without a signal handler. PID 1 ignores SIGTERM by default, so every
`docker stop` waited the full ten-second grace period and then killed the
container (exit 137). That delay was paid on every idle sleep, explicit
stop and Team/Private computer switch, and pushed those requests past the
mobile app's RPC timeout.

The script now traps TERM/INT, forwards the signal to Xvfb and the other
desktop processes, waits for the display to exit and returns 0. Waiting on
the Xvfb process replaces the polling loop so the trap runs immediately.

* fix(computer): install the shutdown handler before any child starts

A stop that arrives while the desktop is still starting is now honoured too: the TERM/INT trap is installed at the top of the script, guards an Xvfb that has not started yet, and the test checks the handler precedes the first background process.

* test(computer): assert the shutdown handler guards an unstarted Xvfb
…lie222#897)

Claude Code OAuth rejects tool names that start with mcp_. Lazy catalog wrappers are now connectors_* while direct mcp__server__tool names stay. Approvals map through a shared prefix and resume legacy mcp_execute_tool envelopes.

Co-authored-by: muurlingjunior <muurlingjunior@users.noreply.github.com>
…laceholders (elie222#912)

Empty or missing label/purpose used to become Code/otp, which agents treated as
tampering. Fail those calls with a clear error and leave valid args unchanged.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* chore(mobile): align Expo 57 patch versions for install --check

Unblock Typecheck by bumping expo and sibling expo-* packages
to the ~57.0.x versions expo install --check expects.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* test(mobile): read scene-lifecycle AppDelegate from the pinned Expo template

The plugin matches exact Expo AppDelegate text. Load it from the
installed template.tgz so a patch bump fails this test if the iOS
template drifts, instead of waiting for a prebuild failure.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…ng row (elie222#924)

* fix(docker): recover a computer whose container stopped under a running row

* fix(docker): only a successful inspection marks a container stopped
* fix(bots): let users change a bot profile picture from settings and chat

Reuse the existing color field for uploaded avatars, validate writes, and
extend update_bot so a bot can set its own picture in the owning space.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(bots): keep Avatar Studio on main; only fill the chat gap

Web settings already ship picture upload via Avatar Studio. Revert the extra Studio testids, Upload-tab E2E, and ui-web parser move so this change stays on update_bot plus mobile rendering of stored pictures.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(mobile): render stored bot avatar shapes and skip unchanged color

Keep the encoded shape index on mobile so a web-selected mascot matches
across surfaces. Omit unchanged color from web profile saves so a legacy
named value cannot fail an unrelated update.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(mobile): use the shared palette for shipped avatar eyes

Amber Gold is darker than the brightness cutoff, so a local heuristic
drew white eyes while web uses the palette dark eyes. Resolve eye color
through the same persona colors as web.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…e222#914)

* fix: strip Markdown from sidebar previews and push notifications

Preview snippets and completion push bodies now use shared plaintext
from Markdown so markers are not shown, applied before truncation.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: keep autolinks and parenthesized links in preview text

plainTextFromMarkdown treated angle-bracket autolinks as HTML and
stopped markdown destinations at the first ')', so previews could go
blank or leak markers. Protect code payloads and match fence closers
by opener length.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: parse preview links in linear time and avoid sentinel collisions

Precompute bracket and parenthesis closers once so unmatched '[' no
longer rescans the rest of the message. Pick a stash mark that is not
already in the source so U+E000 in a message cannot be restored as code.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: choose preview code-stash marks in linear time

unusedMark grew the sentinel by one U+E000 and rescanned the whole
source each time, so a long private-use run was quadratic. One pass
now records the longest run and uses a longer mark.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix: keep escaped markdown punctuation and unsplit preview emoji

Backslash-escaped markers such as \* are now restored as literal
punctuation instead of being eaten by emphasis stripping. Truncation
still uses the 180 UTF-16 budget but drops a trailing high surrogate
so a cut cannot emit a lone pair.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…ie222#915)

* fix(mobile): keep the computer stream across rotating screen URLs

The live computer WebView was keyed on the full capability URL. Each
computer/screenUrl response mints a new token, so preview and full-screen
remounted even while the same computer was still running.

Key and source now follow the stream identity (origin, view vs control)
instead of the rotating token, so takeover/release still reconnects.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(mobile): apply renewed computer screen capabilities before expiry

Token-only URL updates still keep the connected stream. When the held
capability is in the renew window, adopt the new URL so the proxy does
not close the live session when the original token expires.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
…222#916)

* fix(computer): let touch devices paste without a keyboard chord

Add a Paste control on touch computer sessions that uses the existing
clipboard bridge, with a keyboard-focus fallback when the clipboard API
is unavailable.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(computer): prepare touch paste fallback during the tap

Focus the keyboard field before awaiting clipboard so a denied or missing
Clipboard API still has a software keyboard. Park the caret after the
sentinel so OS paste does not send those underscores. Align EventTarget
test doubles so sandbox-supervisor typecheck passes.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* fix(chat): accept free-text as the answer while waiting_input

Composer sends were rejected until the user tapped an offered option.
Treat a normal chat message as answering the pending question so a custom
reply can continue the run.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(chat): continue every waiting run a free-text answer satisfies

Free-text send answered all waiting_input runs but kept only the first in
the group map and omitted runIds from the 1:1 send event, so extra runs
missed immediate continue and same-nonce replay.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* fix(web): open the computer from the Needs you thread card

The takeover card showed Computer / Needs you with no control that
opens the computer. Reuse the existing open-computer path from an
Open button on the card, and cover that path in web E2E.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(web): open Needs you computer cards for group member bots

Open on a group transcript card now targets that member bot through the
same openComputer path, instead of returning because group mode has no
active 1:1 bot. Mark the computer visible before the screen refresh so
the overlay is not blank when the panel starts closed.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(web): ignore stale computer boot results when another open starts

A slower first boot was still clearing the overlay spinner and could
write its error after Open had already targeted a different bot.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* style(web): wrap the computer heartbeat callback for biome

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* fix(web): keep group computer overlay on the selected bot

Ignore peer computer events in a group thread so another member's
takeover cannot rewrite the open overlay, and assert the card Open
path calls computer RPCs for that member bot.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
* feat(bots): let update_bot set notifyOnFinish

Agents could not turn finish notifications off even though the column and
delivery gate already existed. update_bot now accepts notifyOnFinish for
the calling bot, persists it, and returns the stored value.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

* test(bots): expect notifyOnFinish on update_bot schema

The avatar schema test listed exact property names. After rebasing onto
profile-picture support, notifyOnFinish must appear in that list too.

Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Elie Steinbock <elie222@users.noreply.github.com>
- Reuse the shared sandboxIdleMs() helper instead of a second idle-time
  computation with a different default and floor.
- Wait for a reused sandbox to reach running from any non-running status,
  not only from paused or error.
- Mark the workspace dirty on GUI actions so exportWorkspace no longer
  drops click-only and keystroke-only changes.
- Report a timed-out command as exit code 124 instead of letting the HTTP
  abort escape the async generator.
- Clear lastBrowserUris and dirtyWorkspaces on stop and destroy.
CREATEOS_API_KEY and CREATEOS_API_URL were never documented in
.env.example. Keep one name per setting. Also revert the unrelated
computer-status helper extraction.
Live CreateOS reports pausing -> paused and resuming -> running. stop()
returns while the sandbox still reports pausing, so an immediate
provision saw a transitional status, never issued resume, and timed out
after 120 polls. Wait for the sandbox to settle first, then act on the
final status.

Adds a live canary, gated on VERIFY_PROVIDERS and
CREATEOS_SANDBOX_API_KEY, which covers provision, prepare, execute,
observe, connectScreen, GUI-only export, pause, resume, and destroy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.