Skip to content

feat(tiles): move tiles by hand, and detach a tile into its own window - #584

Open
Ark0N wants to merge 1 commit into
masterfrom
feat/tile-gesture-detach
Open

Ark0N wants to merge 1 commit into
masterfrom
feat/tile-gesture-detach

Conversation

@Ark0N

@Ark0N Ark0N commented Oct 10, 2026

Copy link
Copy Markdown
Owner

What this adds

Two things for the tile grid (the Tiles view that shows up to 6 sessions side by side). Meant for a later release.

1. Gesture control moves tiles

With the camera overlay on (instance started with CODEMAN_GESTURE=1, then App Settings → Gesture Control) and the tile grid open:

  • pinch anywhere on a tile and carry it onto another tile: the two swap places
  • carry it onto an empty cell: it moves there
  • carry a session tab onto a tile or an empty cell: it joins the grid there (what a mouse-dragged tab already does)
  • a pinch that barely moves (under 40 px) changes nothing

While carried, the tile dims, a small copy of it follows the hand, and the tile or cell under the hand is highlighted (the mouse drag's own highlight). The hand never moves tiles by itself: the overlay asks tile-grid.js what is under the point and drops through the same paths the mouse header drag uses (dropSessionOnTile / dropSessionOnSlot, so _reorderTiles stays the one move path). Hand and mouse cannot disagree about what a drop does.

2. Detach Tiles (new setting, off by default)

App Settings → Appearance → Detach Tiles (per device, desktop only, beta). With it on:

  • drag a tile's header out of the browser window and let go: the tile opens as a window of its own (the same pop-out as the tab's "Open in a new window"), the tile's size, near where you let go
  • drop it on another Codeman window's tiles instead: it moves into that window's grid and leaves this one
  • drag a popped-out window's title onto any Codeman window's tiles: the session docks there as a tile and the pop-out closes
  • a tile's ⋯ menu gets "Open in a new window" (the keyboard path; it opens over the tile, the tile's size)
  • with gestures: a tile carried out of the grid and let go opens as its own window

A lone or zoomed tile can be dragged out too (it just cannot move inside the grid). With the setting off nothing changes: same draggable rules, same tooltip, no foreign drops taken.

How it works

No server change. A pop-out is just another client of the same session (that is how the tab pop-out already works), and a tile is a TerminalTile in whichever window holds it. A session stays in one place in the browser at a time.

  • Out of the window. The header drag notes the tile's size and where it was grabbed. At dragend, if no drop target took it (dropEffect: 'none') and the drop point is outside the viewport, the tile opens via detachSession(id, placement) after a 120 ms settle. Where a browser reports no drop point (0, 0), the page's own record of the pointer leaving at the viewport edge decides (a later dragover inside clears it). detachSession gained an optional placement (size + screen position); without one the window features are byte-identical to before.
  • Into another window. The drag carries application/x-codeman-tile (never text, so no text field or terminal can take it as typing). Another window's tiles and empty cells accept a drag that is none of their own and carries that type, dock it through the target's own drop, and post tile-adopted on the existing codeman-windows BroadcastChannel. Only the window the tile was dragged from acts on that (its drag still settling, or ended within 5 s): it drops its copy without moving the keyboard, and if that was its last tile it goes to the welcome screen, never onto that session (two windows sizing one PTY garble it). A claim during the 120 ms settle cancels the pop-out.
  • A pop-out back into a grid. Docking a popped-out session takes it back first (a detached session never mounts), then sends close-request. The pop-out is now "released": it says redocked at once, stops answering roll-calls and closes; one a script may not close (opened by typing its URL) shows "Session moved" instead of staying a pop-out that dashboards would mark detached again. The docking window ignores that pop-out's detached answers for 2 s, which would otherwise take the new tile straight back off.

Known limits

  • Multi-monitor placement: Chrome keeps a script-opened window on the opener's screen unless the page holds the Window Management permission, so a drop on a second monitor opens at the edge of this screen; drag the window across from there. Wayland ignores window positions entirely.
  • Escape outside the window: browsers report "cancelled with Escape" and "dropped on the desktop" the same way, so Escape with the pointer outside the window pops the tile out. Bring the pointer back in to cancel.
  • Gesture pop-out needs pop-ups allowed for the site: a hand is not a click, so a pop-up blocker refuses it. Codeman's existing toast says so and the tile stays.
  • Another window opened before the switch picks up Detach Tiles on reload (settings are cached per page). Pop-out windows follow it live.
  • CLAUDE.md: the tile paragraph gets one rule line here. If the in-flight move of the frontend rules into src/web/CLAUDE.md lands first, that line moves with it.

Files

  • src/web/public/tile-grid.js: the Detach Tiles section (setting check, tear-off tracking and settle, foreign docking, tile-adopted handling, the solo title handle) and the gesture helpers (tileAtPoint, canGrabTile, tileDropTargetAt, dropOnTileTarget, isOverTileGrid, detachTileAtPoint); the header drag and its tooltip know the setting
  • src/web/public/app.js: detachSession placement, tile-adopted on the window channel, the released pop-out
  • src/web/public/tab-rail-resize.js: the tile's ⋯ menu entry
  • src/web/public/settings-ui.js, index.html, styles.css, i18n.js: the setting (display key, stripped from the PUT, absent from the schema), its row and Help row, the solo title cursor, zh-CN for every new string
  • packages/gesture-control/src/codeman/entry.ts + the rebuilt src/web/public/gesture/gesture-codeman.js: tile grabs, tab drops onto the grid, pull-out
  • docs: docs/architecture-invariants.md (Tile grid → Detach Tiles, gesture source), the gesture README, one line in CLAUDE.md
  • .changeset/tile-gesture-detach.md (minor)

Testing

  • test/tile-grid-detach.test.ts (new, 33 tests, the real tile code in the shared vm harness): off changes nothing; out of the window opens a placed pop-out after the settle; inside or taken by a target opens nothing; the edge-leave fallback; listeners removed on drag end and on a grid close mid-drag; the real detachSession path removes the tile; tile-adopted handling (source only, own nonce ignored, stale drags ignored, last tile to the welcome screen, claim cancels a settling pop-out); foreign drops on a tile and an empty cell; a popped-out session docked with its stale answers ignored; unknown ids dock nowhere; own drags never read as foreign; window features; the ⋯ menu (JSDOM); a released pop-out stops answering roll-calls; the gesture helpers; the setting's per-device plumbing
  • test/tile-grid-i18n.test.ts extended: the two new header hints are harvested from the real code and translate with the existing state-plus-duration pattern; the Help row, the settings row and the pop-out strings translate
  • npm test (the CI gate): 10,388 passed, 12 skipped, 1 failed. The failure is test/spreadsheet-preview-worker.test.ts > "refuses a shared string of 2,000,001 empty runs before ExcelJS loads": it hit its 60 s timeout on a loaded machine (load average ~8) and passes on its own in 50 s. This PR touches no spreadsheet code
  • npm run typecheck, lint, format:check, check:frontend-syntax, check:public-assets, and tsc --noEmit in packages/gesture-control: clean
  • End to end against an isolated beta instance (own CODEMAN_INSTANCE, four throwaway shell sessions, Playwright + Chromium, two dashboard pages talking over the real BroadcastChannel): 27/27. A real mouse drag of a header out of the window opened /session/<id> and the tile left the grid; that pop-out's title was a drag handle, and dropping it on the second dashboard's empty cell docked it, closed the pop-out and un-marked it in the first dashboard; a tile whose drag ended as taken elsewhere left the first grid when the second docked it; through the live gesture overlay a carried tile highlighted its target, the drop swapped the two tiles, a stray pinch moved nothing, and a tile carried above the grid opened as a pop-out; with the setting off a drag out of the window opened nothing and the tooltip was unchanged; the settings row shows on desktop, off.
  • Not covered by automation: a physical drag between two separate OS windows (Playwright cannot drag across windows, so the receiving side was driven with a real DataTransfer drop and the sending side with the dragend a browser sends), and a real camera at the desk.

Gesture control now moves tiles: with the tile grid open, pinch anywhere
on a tile and carry it onto another tile (they swap) or an empty cell (it
moves there); a session tab carried onto the grid joins it there. The
overlay asks tile-grid.js what is under the hand (tileAtPoint,
tileDropTargetAt) and drops through the mouse drag's own paths
(dropSessionOnTile / dropSessionOnSlot, so _reorderTiles stays the one
move path).

Detach Tiles (App Settings > Appearance, per device, OFF by default,
desktop only):
- a tile's header let go outside the browser window opens the session as
  a pop-out (detachSession, which now takes an optional size and screen
  position), the tile's size, near the drop point, after a 120 ms settle;
- dropped on another Codeman window's tiles it moves there: that window
  docks it through the target's own drop and posts 'tile-adopted' on the
  window channel, and only the window it was dragged from lets go of its
  copy (a last tile goes to the welcome screen, never onto that session);
- a pop-out's title drags back onto any grid: the session is taken back,
  the pop-out is asked to close and is released (stops answering
  roll-calls; one a script cannot close says where the session went), and
  its stale 'detached' answers are ignored for 2 s;
- a tile's menu offers "Open in a new window", and a hand that lets a tile
  go outside the grid pops it out too.

With the setting off nothing changes. No server change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Ark0N
Ark0N force-pushed the feat/tile-gesture-detach branch from df636c5 to 2c55fe9 Compare October 10, 2026 05:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants