Skip to content

feat(extension): attribute agent-opened tabs to the task that opened them - #297

Open
lyingbug wants to merge 1 commit into
Tencent:mainfrom
lyingbug:feat/task-popup-attribution
Open

lyingbug wants to merge 1 commit into
Tencent:mainfrom
lyingbug:feat/task-popup-attribution

Conversation

@lyingbug

Copy link
Copy Markdown
Contributor

Background

Tab control has two shapes today. A local session may drive any tab inside its Agent Window; a remote session must own each tab explicitly, created through tab_create or borrowed with the user's confirmation (authoriseAgentTab, tabs.ts).

That leaves a gap for tabs a page opens by itself. When an agent clicks a target="_blank" link, or the page calls window.open, Chrome creates the tab and nothing attributes it to the task that caused it:

  • A remote session sees it as scope=user. tab_select, snapshot and every page operation return permission_denied, and the only way forward is tab_borrow, which waits for the user to approve an overlay in that page — for an unattended task, the turn simply stalls. Links that open in a new tab are ordinary on search results, admin consoles, attachment links and OAuth flows, so this is not a rare corner.
  • A local session is unaffected for a new tab in the same window, but a popup that opens its own window is denied with agent_window_scope, because it is no longer inside the Agent Window.

docs/remote-extension-connection.md records the gap: "Automatic popup authorization is outside this version's scope." This PR proposes closing it without weakening the authorization model.

Change

The dispatcher wraps the tools whose page input can open a tab (click, press_key, evaluate, fill, select, and the three navigations) and watches Chrome's webNavigation.onCreatedNavigationTarget while the action runs.

A target is attributed only when Chrome reports it, during the action, from a tab the session already controls, and no other session owns it. rel="noopener" targets and nested popups qualify, because the attribution comes from Chrome's event rather than from openerTabId. A popup that opened its own window is moved into the Agent Window — which is the part that also fixes the local agent_window_scope denial.

Deliberately unchanged:

  • Window membership and openerTabId never authorize a tab on their own. A tab the user drags into the Agent Window stays unauthorized for a remote session.
  • A target Chrome reports without a source, or after the action ended, keeps the ordinary borrow flow with its user confirmation.
  • One task cannot take a tab another task owns.
  • A tab already inside the Agent Window still cannot be borrowed in place; that rule is untouched here.

The attribution window closes as soon as the action ends, with one exception: Chrome can deliver the navigation-target event just after acknowledging the input that caused it. The action therefore waits up to 100 ms — but only when a tab actually appeared while it ran, observed through tabs.onCreated. An action that opened nothing pays nothing, which keeps this off the hot path for ordinary clicks and typing.

Rationale for the security shape

The agent already drives the source page and could navigate that same tab to the same URL, so attributing a target it just caused grants nothing it could not already reach. The alternative is not more safety: it asks the user to approve something the agent visibly just did, and that kind of reflexive prompt is what erodes the confirmations that matter — borrowing a tab the user already had open and signed into.

Validation

  • pnpm --filter @browser-skill/extension test: 1867 passed, 103 skipped, including 5 new cases — noopener and nested targets plus the popup-window move, a local session's popup window, refusing another task's tab and an unauthorized source, the event tail being paid only when a tab appeared, and listener cleanup when the action throws.
  • pnpm --filter @browser-skill/extension compile passes.
  • Not exercised against a live browser in this branch.

Notes

The 100 ms tail is a heuristic for one Chrome ordering detail; if you know a signal that makes it unnecessary, I would rather use that. I am also happy to split the popup-window move into its own change if you prefer to take the local fix first.

…them

A page can open a tab by itself, and nothing attributes it to the task whose
action caused it. A remote session sees `scope=user` and can only reach it
through `tab_borrow`, which waits for the user to approve an overlay in that
page, so an unattended task stalls on an ordinary `target="_blank"` link. A
local session is fine with a new tab in the same window, but a popup that
opens its own window falls outside the Agent Window and is denied.

Watch `webNavigation.onCreatedNavigationTarget` for the duration of the tools
whose page input can open a tab. Attribute a target only when Chrome reports
it, during the action, from a tab this session already controls, and no other
session owns it; move a popup that opened its own window into the Agent Window,
which is what makes it reachable for a local session too.

Window membership and `openerTabId` still authorize nothing on their own, a
target reported without a source or after the action keeps the borrow flow,
and a tab already inside the Agent Window still cannot be borrowed in place.
Chrome can deliver the event just after acknowledging the input, so the action
waits up to 100 ms for it, but only when `tabs.onCreated` showed that a tab
actually appeared while it ran.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant