Device Leases in Postgres: Stop Parallel Agents Fighting Over Emulators

Running pkill -f maestro once wiped out an Android emulator too, because the virtual device happened to be called maestro_e2e. It was one of three silent collisions in a single session. The cure was a Postgres table, a 243-line bash file and three added lines per test runner.

The commit that introduced the registry explains the background. Nine worktrees existed and sessions ran concurrently, yet every runner behaved as if it were alone. The three collisions:

• pkill -f "maestro" also took down the Android emulator, since the AVD name maestro_e2e matched.
• Simulators got booted, erased and deleted without any check on whether another session was running tests on them.
• When two sessions share one device, the failure looks like a bug in the product. That's the expensive collision, since nothing in the symptom points at the real cause.

Today the repo holds eleven worktrees. Each Claude Code session gets its own, many are active simultaneously, and all of them draw on the same two Android emulators, a real Galaxy Z Fold accessed via Samsung's Remote Test Lab, and two iOS simulators. No mechanism kept them apart.

One table, and the primary key is the lock

The registry is one Postgres table in a dedicated database called yarina_devtools. It's not a schema inside the app's database on purpose: it should stay out of app migrations, dumps and backups, and dropping it should never be scary.

CREATE TABLE IF NOT EXISTS device_lease (
  device_id   text PRIMARY KEY,
  platform    text NOT NULL CHECK (platform IN ('ios', 'android')),
  worktree    text NOT NULL,
  session_id  text NOT NULL,
  holder_pid  integer NOT NULL,
  purpose     text NOT NULL DEFAULT '',
  acquired_at timestamptz NOT NULL DEFAULT now(),
  expires_at  timestamptz NOT NULL
);

The primary key does all the work for atomicity. A claim is one statement: INSERT … ON CONFLICT (device_id) DO UPDATE … WHERE the current row is owned by the same session or has expired, then RETURNING session_id. Concurrent claimants are serialized by Postgres on that key, so among N racers precisely one succeeds. Had the design used a history table with an active flag, "is this device free?" would become a query, and that is where a race between claimants would creep in.

Before touching a device, each runner does the same three things:

source "$REPO_ROOT/e2e/scripts/device-lease.sh"
trap 'lease_release_all' EXIT INT TERM
lease_claim "$DEVICE" android "android $SUITE" || exit 1

A refused claim prints who holds the device (worktree, session, purpose, how long) along with the command to clear it. For liveness the script runs kill -0 against the holder's PID. Postgres has no way to know whether a process is running, but everything here is on one machine, so the caller can ask. A 4-hour TTL covers the case of a recycled PID. The table is created on the first claim; creating it at session start was tried and dropped, since most sessions never use a device.

Three bugs the tests found

Real use would never have exposed these.

psql prints a command tag. A refused claim returns zero rows, and that is the refusal. Without -q, though, psql printed INSERT 0 0 on stdout, so the empty-output check saw text and every contention test read as a success. The contention test caught it, and day-to-day use would have hidden it.

Checking liveness before claiming is racy. The first version looked up the incumbent's PID in one query and passed the result to the insert. Eight racing claimants each saw no incumbent, each concluded the device was free, and all eight "won". The order is reversed now: try the claim, never taking from a live holder, and only after a refusal look at the incumbent and delete it with a compare-and-swap on (holder_pid, acquired_at), then retry once. Out of eight, one wins.

A CLI claim that records its own PID dies the moment it exits. Manual claims therefore record holder_pid 0, meaning no owning process, and the TTL decides when they lapse.

A runner added a fourth problem. With exec maestro … the shell process is replaced, so the EXIT trap can't run and the lease stays after the run ends. Invoke maestro as a normal command. The trap is essential: if the lease is freed only when everything goes well, the device stays stuck after failed runs, and those are the runs most likely to be started again immediately.

Fail closed and state the gaps

When Postgres is unreachable, the runner refuses to continue and says why. Quietly degrading to a no-op is worse than having no registry, because the operator believes they're protected when they aren't. YARINA_DEVICES_OPTIONAL=1 is the documented way to go ahead unguarded.

Calling maestro test --device X directly sidesteps leases completely. A wrapper shim over maestro, simctl and adb was weighed and rejected as intrusive and easy to bypass whenever something invokes the real binary. So the runners form the boundary, and the rule sits in CLAUDE.md, the single file each session reads.

A lease doesn't make a green run true

Sharing devices forced two more checks.

Verify the build by hash. Three green runs had actually exercised an APK installed by another session. The replacement run compared the hash of the APK on the device (pm path, then shasum) with the local build, once before and once after. This is now the standing rule for any shared device, because a passing flow tells you nothing about code that isn't on the device.

Watch the machine's load. Code that passed with a load near 8 failed near 20 on retry timeouts, with no changes. Two simulators and an Android emulator together lifted the load average to 94 and crashed SpringBoard 16 times, so the iOS runner now declines to start over a load threshold. Giving the emulator more RAM pushed swap to 93%, and the OS then killed the emulator partway through a run, another thing that looks like flakiness.

Applying it in your repo

1. Use one table with the device ID as primary key, and claim with a single INSERT … ON CONFLICT … WHERE … RETURNING.
2. Release from a trap on EXIT INT TERM, and keep exec out of the runner.
3. Refuse to run when the store is down, with one explicit escape hatch.
4. Run a contention test with N racers before relying on the lock, assert on empty output, and pass -q to psql.
5. Before believing a green run on a shared machine, hash the APK on the device and look at uptime.
6. Write the rule where every session reads it, because a registry nobody knows about gets bypassed by default.

What to remember

The takeaways: parallel agents need a lock around physical devices, because two sessions on one emulator yield failures that pass for product bugs. The lock belongs in the primary key, as one atomic statement without a read-then-write step. A green contention test can lie, and INSERT 0 0 counted as success until someone checked. Fail closed and name the holder, since a silent no-op is worse than having no registry. Finally, a lease isn't the last line of defense: hash the build, watch the load, then trust the run.