remote trackpad and keyboard server
git clone https://git.lucas.co/cce-remote.git
CLAUDE.md (20.7K)
1 # CLAUDE.md
2
3 > This is the `cce-remote` crate, inside the larger **`cce` Cargo workspace** —
4 > read `../cce-compositor/WORKSPACE.md` first for the multi-repo layout, the
5 > standalone-build rule, `ccebuild`, and the `cce-ui` toolkit. This file covers only
6 > what is specific to this crate.
7
8 `cce-remote` turns a phone into a trackpad, keyboard, window switcher and live window
9 viewer for the cce desktop. It is **the odd crate in this workspace**: not a compositor
10 and not a Wayland GUI client. It has no `cce-ui` dependency, draws nothing, and opens no
11 Wayland surface of its own (it *does* connect to Wayland, but only as a screencopy
12 client). It is a headless LAN server — HTTP + WebSocket in, compositor control socket
13 out — and its entire user interface is one hand-written `index.html` compiled into the
14 binary with `include_str!`.
15
16 Mirroring a sibling app for structure is therefore the wrong instinct here. There is no
17 `Application` trait, no `Message` enum, no widget tree. Four files:
18
19 - **`src/main.rs`** — PIN auth + rate limiting, the HTTP/WS dispatch, the
20 frame→command translator.
21 - **`src/stream.rs`** — live-view delivery: the latest-wins `Slot`, the ack-clocked
22 sender and its adaptation ladder, and the producer that picks a frame source.
23 - **`src/screencopy.rs`** — a persistent `wlr-screencopy` client (frame source #2), and
24 `downscale_encode`, shared by both raw frame sources.
25 - **`src/winstream.rs`** — consumer of the compositor's window-stream socket (source #1).
26
27 ## The invariant: the control socket is a full-privilege injection channel
28
29 Everything this server does, it does by writing lines to
30 `/tmp/cce-{WAYLAND_DISPLAY}.sock` — the same channel `ccectl` uses. Anything that can
31 put a line on that socket can move the pointer, click, and **type arbitrary keystrokes
32 into whatever the user has focused**. There is no sandbox between a WebSocket frame and
33 the user's session except the code in this crate.
34
35 So the discipline is: **nothing from the network is ever forwarded raw.** `translate()`
36 is a whitelist that matches a fixed set of shapes and *rebuilds* the command string from
37 re-parsed values — an unrecognized verb returns `None` and the frame is dropped.
38 Numbers go through `parse::<f64>()`/`parse::<u32>()`, so they cannot smuggle a newline
39 and inject a second command. The one place a caller-supplied *string* reaches a command
40 (`wf <target>` → `focus-window`) is gated on `safe_token`. Named actions are whitelisted
41 individually by name — note that `cmd` deliberately matches `"restart-compositor"`
42 against a literal rather than passing the name through, and that shape is the point.
43
44 Two things to keep in mind when adding a message:
45
46 - **`translate()` is the single chokepoint — keep it that way.** It returns a *list*
47 of commands so a verb can expand to several without the expansion living inline at
48 the call site (the retired view-tap did: absolute move, then click — view-mode taps
49 are plain trackpad clicks since 2026-08-23, and the verbs left the whitelist rather
50 than lingering as unused injection surface). The only frames still handled in
51 `handle_ws` are `wl` and `pl`, which produce a *reply* and send fixed commands
52 carrying no caller-supplied content. A new message that carries any part of the
53 frame into a command belongs in `translate()`, where the tests can see it.
54 - **`cmd restart-compositor` restarts the user's whole session** from a phone, behind
55 nothing but a client-side `confirm()`. Compositor-side it writes
56 `/tmp/cce-restart-requested-$USER` and exits cleanly (state is saved, and
57 `cce-display-manager`'s daemon relaunches greeter-free). Anything added to that `cmd`
58 whitelist gets the same reach.
59
60 ### What the PIN does and does not buy
61
62 A persistent 6-digit PIN (`~/.config/cce/cce-remote.pin`, 0600, generated from
63 `/dev/urandom` on first run, honoring `XDG_CONFIG_HOME`) must arrive as the **first**
64 WebSocket frame or the connection closes — with a 10s read timeout so unauthenticated
65 peers can't sit on a socket. The HTTP frame endpoints are gated separately, and
66 differently, because of a browser constraint that shaped them when the page still
67 used both: `/shot` takes an `X-Pin` header, but `/stream` accepts `?pin=` in the query
68 string, because an `<img src>` cannot carry headers. The page uses NEITHER today — the
69 live view rides `/wstream` — so both are debug endpoints now, and the split survives
70 for the curl recipes below rather than for a browser.
71
72 All three gates are pure functions — `auth_frame_ok`, `header_pin_ok`, `query_pin_ok`,
73 all over `pin_matches` — so they are unit-tested rather than only reachable through a
74 socket. `pin_matches` refuses an **empty** PIN outright: `load_or_create_pin`
75 regenerates on an empty file so it should be unreachable, but that is a property of a
76 *different* function, and if it lapsed, a bare `X-Pin:` header would authenticate
77 everything. Gate on the dangerous state, don't trust the caller.
78
79 ### The rate limiter is what makes 20 bits a credential
80
81 A 6-digit PIN is ~20 bits compared with `==`. What keeps that from being walked in an
82 afternoon is not the comparison, it is `RateLimiter`: a **per-source-IP token bucket
83 over failed attempts**, 5 back-to-back then one recovered per 30s. That caps sustained
84 guessing at ~2/min, which turns a couple of hours into the order of a year. All three
85 gates consult it, and an unresolvable peer address is refused rather than exempted.
86
87 Three properties it must keep, each with a test:
88
89 - **Only failures are charged, and a success clears the record.** The page reconnects
90 its stream on every hiccup — a WS close, the no-frame watchdog — each time presenting
91 a correct PIN. If those consumed budget, a working client would throttle itself off.
92 - **Refill caps at the burst.** Otherwise an idle attacker banks attempts and the limit
93 is only an average. Note the test asserts this on `refilled()` *directly*: going
94 through the public API hides a missing cap, because `record_failure` prunes recovered
95 peers and re-creates them at full.
96 - **The table cannot grow without bound**, or the limiter becomes its own
97 memory-exhaustion vector. Recovered peers are pruned on write (a full bucket is
98 indistinguishable from an absent one) with a hard cap behind that, evicting whoever is
99 closest to recovered.
100
101 On the WS side, a rate-limited connection is closed **without** sending `auth fail` —
102 that message makes the page discard its stored PIN and prompt, so sending it would
103 punish a correctly-paired client for someone else's guessing from the same address. The
104 page reconnects on close and succeeds once the bucket refills. HTTP answers `429` with
105 `Retry-After`.
106
107 Be honest about what is left rather than treating the PIN as security: it still travels
108 over **plain HTTP on 0.0.0.0**, is compared non-constant-time, is cached in
109 `localStorage`, and for `/stream` it rides in a URL, where it lands in any proxy or
110 history that sees it. Limiting is per-IP, so a peer with many addresses gets many
111 budgets. It is pairing — it stops other devices on a trusted LAN from steering the
112 desktop by accident, and now also stops casual brute force. It is not a defense against
113 someone who is on that network on purpose. For a hostile network the answer is a tunnel.
114
115 ## Framing: the control socket is one-shot
116
117 `control_command()` opens a **fresh `UnixStream` per command** and reads the reply to
118 EOF. That is not wasteful, it is the protocol: the compositor's IPC server is
119 read → reply → close. An earlier version held one persistent stream and silently raced
120 reconnects, dropping commands; reading to EOF is also what makes multi-line replies like
121 `windows --json` work at all.
122
123 The page therefore does the coalescing: `queueFlush()` batches pointer deltas on a 12ms
124 timer so a fast drag becomes ~80 commands/sec, not one per touch event. One thread is
125 spawned per accepted connection, uncapped, and a `/stream` connection holds its thread
126 for the life of the stream.
127
128 ## The live view: latest-wins delivery, three frame sources
129
130 **Delivery and capture are separate concerns since the 2026-08-22 rework.** The
131 original MJPEG path pushed every frame, in order, into a blocking TCP write; nothing on
132 this side ever dropped one, so the kernel's send buffer (~10-30 frames) became a queue,
133 and the moment wifi throughput dipped below the frame rate the view fell seconds behind
134 and never recovered — "fine at first, unusable after a short time".
135
136 The delivery design (`stream.rs`) makes that failure structurally impossible:
137
138 - A **`Slot`** holds only the newest frame; the producer overwrites it. Overwriting IS
139 the frame-dropping — stale frames cease to exist before they cost encode or network.
140 - The page's live view rides **`/wstream`**, a dedicated WebSocket (same `auth <pin>`
141 first-frame gate): the server sends one frame, the page renders it and acks `n`, and
142 only then does the newest frame go out. **At most one frame is ever in flight**, so a
143 degraded link costs frame *rate*, never accumulating latency. The ack is sent after
144 `drawImage`, not on receipt — so the measured send→ack time covers network + decode +
145 paint, which is what the user experiences.
146 - That measurement drives an **adaptation ladder** (`LADDER`/`adapt()`): resolution up
147 to 1400px edge when the link is fast, downgrades immediate, upgrades requiring
148 sustained headroom. Encoding happens per *sent* frame at the chosen level. The ladder
149 alternates rather than sacrificing one axis first — `(1400,68) → (1120,68) →
150 (1120,55) → (840,58) → (840,46) → (560,48)`, a size drop first, and quality rising
151 again where size falls. `ladder_prefers_resolution_over_quality` does NOT assert the
152 preference its name claims: it only checks that the edge never increases, which a
153 ladder that dropped size at every step would also satisfy.
154 - `/wstream` is deliberately a **separate socket from the input WS**: frames are
155 30-150KB and input events are bytes; one TCP stream would head-of-line-block pointer
156 motion behind every frame.
157 - `/stream` (MJPEG over HTTP) survives as the **curl-debuggable endpoint**, thin over
158 the same slot at fixed 560/q60. Without acks its TCP buffer can still hold a few
159 frames — fine for debugging, which is all it is for now.
160
161 Every accepted socket gets `TCP_NODELAY` — before the rework nothing set it, so Nagle
162 was batching tiny input events behind delayed ACKs.
163
164 The ceiling above this design is hardware H.264 + WebCodecs/WebRTC (~5-10× fewer bytes),
165 at the cost of VAAPI/GStreamer deps and Safari codec quirks. Ack-clocked adaptive JPEG
166 is the right cost/benefit for a single-window view on a LAN; revisit only if it proves
167 bandwidth-starved in practice.
168
169 ### The three frame sources
170
171 `spawn_producer` tries each in turn. All three are live code — the fallbacks exist
172 because the first two have real preconditions.
173
174 1. **`winstream`** — subscribe `window focused` on `/tmp/cce-stream-{WAYLAND_DISPLAY}.sock`
175 and read `frame <w> <h> <len>` + packed RGBA. Best source: damage is *per window*, it
176 follows focus server-side, it streams occluded and off-viewport windows, and a truly
177 idle window sends nothing but a ≤15s keepalive. Requires a compositor built with
178 `stream_server.rs`; against an older running `cce-fx` the connect fails and we drop to
179 screencopy, which is why this is a fallback chain and not a choice.
180 2. **`screencopy`** — one long-lived `wlr-screencopy` connection, per-frame
181 `capture_output_region` of the focused window's rect, throttled by `copy_with_damage`.
182 ~11 fps active. Its damage gate is **per output, not per region**, so an idle window
183 still wakes on unrelated screen activity.
184 3. **`grim`** — fork per frame, `-s 0.5 -q 65`. ~2.5 fps. The floor.
185
186 Frames are box-downscaled and JPEG'd by the shared `downscale_encode`, at whatever
187 (edge, quality) the ladder picked for the link — not a fixed size anymore.
188
189 **The cursor differs between sources, and the page compensates for the worst case.**
190 Compositor window-stream frames are surface textures with no cursor composited, so the
191 page draws its own cyan ring. Since that stream is *damage-driven*, moving the pointer
192 produces no repaint at all — which is why the marker is predicted client-side from the
193 finger delta (same `ACCEL` as the sent move) and only *reconciled* by a 120ms
194 `pl`/`ploc` poll. Polling alone updated it 4×/s and felt broken. The screencopy path
195 passes `overlay_cursor=1`, so on that fallback you see the real cursor *and* the ring.
196
197 `/shot` (the one-shot PNG) screenshots through the compositor and then **deletes the
198 file** — verified: a `/shot` leaves nothing new in `~/Pictures/screenshots`. Remote
199 viewing must not accumulate captures on disk.
200
201 ## Geometry assumes one output at 0,0
202
203 The region passed to screencopy is in output-local logical coordinates, and the window
204 rects from `windows --json` are in layout coordinates. Those are the same number only
205 because there is a single output sitting at the origin — true for this DE's eDP-1 setup,
206 and the same assumption `grim` ran under. Multi-output would need a real mapping here.
207
208 The cursor marker (`placeMarker`) maps the other way, through the `object-fit:
209 contain` letterbox, and stays correct under pinch-zoom only because the zoom is a
210 **uniform** CSS transform on an ancestor, so `getBoundingClientRect()` already
211 reflects it. (Tap-to-spot mapping is gone: view-mode input is trackpad-identical —
212 taps click where the cursor is.)
213
214 ## `index.html` is the client, and iOS Safari shaped most of it
215
216 The page is versioned here and baked in at compile time, so **a UI change needs a
217 rebuild, reinstall and restart of the server** — there is no asset path to edit live. It
218 is vanilla JS, no build step, no dependencies.
219
220 Four of its non-obvious constructs are scar tissue. Do not "clean them up":
221
222 - **The hidden textarea keeps sentinel padding** (`········`, cursor at the end, re-armed
223 on focus plus a 1s drift-repair timer). iOS never fires `deleteContentBackward` on an
224 empty field, so without something to delete, backspace silently does nothing.
225 `beforeinput` is used throughout because iOS `keydown` reports keyCode 229.
226 - **The zoom/pan transform lives on `#screenwrap`, not the frame element.** Uniform
227 ancestor transform means `getBoundingClientRect` reflects it, keeping the cursor
228 ring correctly placed while zoomed. (It also used to dodge an iOS bug where a
229 transformed multipart-MJPEG `<img>` stopped repainting; the view is a `<canvas>`
230 since the 2026-08-22 rework, but the structure stays.)
231 - **The `overflow: hidden` clip lives on `#pad`, the non-transformed ancestor.** A clip
232 on the transformed element scales with its own content and clips nothing.
233 - **The stream self-heals: reconnect on WS close plus a 30s no-frame watchdog** (the
234 frame sources force keepalives ≤20s, so 30s of silence is a dead connection, not an
235 idle window). One guard worth keeping: the page only auto-reconnects `/wstream` if
236 that connection *paired successfully* — retry-looping a stale PIN would feed the
237 rate limiter and lock the phone's address out of the input socket too.
238
239 `SCROLL = 0.8`, not the 0.045 it started as: axis values reach clients as surface-px
240 deltas, so near-unity is the trackpad-like 1:1 feel. A 300px swipe used to scroll one line.
241
242 ## Verifying
243
244 The awkward part: **there is no WebSocket client on this machine** (no `websocat`, no
245 `wscat`, no python `websockets`), so the WS path — which is most of the logic — can only
246 be driven from a real phone, or by writing a throwaway client.
247
248 The pure functions are the exception, and they are where the crate's invariants are
249 actually enforced, so they carry all the tests (`cargo test -p cce-remote`, 25 of them:
250 19 in `main.rs`, 6 in `stream.rs`) — `translate()` for what a paired client may say, the
251 three PIN gates for who is paired at all, and in `stream.rs` the `Slot`'s latest-wins
252 semantics plus `adapt()`/`LADDER`. They cover the accepted shapes and — more to the point —
253 everything that
254 must be refused: unknown verbs *including the compositor's own command names*,
255 malformed and missing arguments, `wf` targets outside `safe_token`, unwhitelisted `cmd`
256 names, and the property that no input can make the output span two lines (an embedded
257 newline would be a second command, since `control_command` appends one). Extend them
258 when you touch the whitelist; they are much cheaper than the phone.
259
260 One of them, `non_finite_coordinates_are_dropped`, guards a hole that was live until
261 2026-08-22: `f64::from_str` accepts `"NaN"`/`"inf"` and `{:.2}` prints them straight back,
262 so `m NaN 1` used to reach the compositor's pointer math verbatim. `parse().ok()` is not
263 sufficient validation for a float — hence `finite()`.
264
265 What can be checked from the desktop, and is confirmed working:
266
267 ```sh
268 curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:17017/ # 200, the page
269 curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:17017/shot # 403, no PIN
270 curl -D- -o /tmp/shot.png -H "X-Pin: $(cat ~/.config/cce/cce-remote.pin)" \
271 http://127.0.0.1:17017/shot # 200 + X-Win rect
272 ```
273
274 `X-Win: <id> <x> <y> <w> <h>` in that reply is the focused window's layout rect — the
275 same rect the frame sources capture and the page places its cursor ring inside, so it
276 is the quickest check that focus resolution and geometry agree. (The page does not read
277 this header; nothing in the page fetches `/shot` at all.)
278
279 **Injected input lands in the live session** — pointer moves steer the user's real
280 cursor and keystrokes go into whatever they have focused. To exercise the input path
281 safely, point the server at the **shadow session** instead: run it with
282 `WAYLAND_DISPLAY=` set to the shadow display (see `cce-shadow` in
283 `../cce-compositor/CLAUDE.md`) on a spare port. Both sockets this crate needs exist
284 there — `/tmp/cce-{display}.sock` and `/tmp/cce-stream-{display}.sock` — so even the
285 live-view path is reachable. Note the display name is read from the environment at
286 startup, so a server is bound to whichever session launched it for its whole life.
287
288 ## Build and lifecycle
289
290 `make install` → `ccebuild install --no-build cce-remote`. Never hand-list binaries in
291 the Makefile — `cargo metadata` already knows them. No `Cargo.lock` is tracked here, so
292 dependency changes need no lockfile refresh.
293
294 **Committing is not publishing — pushing is.** This directory is its own git repository
295 whose `origin` is the local *bare* repo `~/git/cce-remote.git` (a real, pushable
296 remote). `published` is the old fetch-only static mirror
297 `https://git.lucas.co/cce-remote.git`, kept for reference; it never accepted a push
298 (dumb HTTP, no receive-pack) and that is exactly why the bare layer exists — see
299 `~/.local/bin/git-bare-sync.sh`. `repos.conf` lists the **bare** path, and
300 `gitsite.timer` republishes when a listed bare repo's HEAD moves. So the chain is:
301
302 ```sh
303 git commit ... # local only
304 git push origin master # this is the publishing step
305 # gitsite.timer then mirrors it to git.lucas.co
306 ```
307
308 An unpushed commit looks published on this machine and is not on the site. That gap was
309 workspace-wide on 2026-09-18 — 21 crates held unpushed commits, because
310 `git-bare-sync.sh` (the bulk pusher) read `repos.conf` field 2 as a work tree when it
311 holds the bare path, and silently skipped every repo. It is fixed and now versioned in
312 the gitsite repo, so `git-bare-sync.sh` pushes the whole set in one go; the backlog
313 stands until someone runs it.
314
315 **The server does not run in the foreground — it is a user service.** `cce-remote.service`
316 ships from this crate root and is installed by `ccebuild` (classified as a user unit by
317 its `WantedBy=cce-session.target`), so it starts with the session and inherits the
318 session's `WAYLAND_DISPLAY` — which is the point, since that variable is read once at
319 startup and fixes which session the process can drive for its whole life.
320
321 Because installs unlink-before-write, the running process stays on the old inode after
322 `ccebuild install` and keeps serving the old code until restarted:
323
324 ```sh
325 ccebuild restart # picks it up automatically — see below
326 systemctl --user restart cce-remote
327 ```
328
329 `ccebuild restart` does reach it, but not because the unit ships from here: it
330 enumerates *running* user services matching `^(cce|gpu-watcher)` and restarts the ones
331 whose `/proc/<pid>/exe` reads `(deleted)`. Being named `cce-remote.service` is the whole
332 qualification. Editing `index.html` counts as a code change for this purpose — it is
333 `include_str!`'d, so the page only updates once the binary is rebuilt, reinstalled *and*
334 the service restarted.
335
336 The unit was unversioned until 2026-08-22 — a hand-written file that existed only in
337 `~/.config/systemd/user/`, in no repo, and so lost on a fresh clone with nothing here to
338 recreate it. Same failure the `.desktop` entries and `cce-keyring-selftest` had before
339 they were moved in-repo. It now installs to that same path from this crate.