Screenshot Evidence for AI Agents: Full-Page Capture, Regions and Diffs

Getting an agent to install Playwright and save a PNG takes a handful of lines. Nobody struggles with that anymore. The trouble starts when those images become part of a production workflow and somebody has to rely on them.

At that point an agent has to prove that a tall page was captured to the very bottom. It has to cut the result into pieces a vision model can actually read, keep evidence that never changes afterwards, compare two releases without layout shifts producing false alarms, run checks on a schedule, and stay inside a budget when nobody is watching.

A managed screenshot API with a remote MCP server is one way to package all of that, for public pages or pages the caller is authorized to see. It isn't meant to replace plain Playwright when a throwaway screenshot is enough. Its job is to serve as the evidence layer for workflows that repeat.

Capturing a page in a defined state

A URL alone doesn't pin down what a screenshot shows. Desktop, tablet and phone widths differ, so do light and dark themes, and so does a page after a click. A capture service therefore needs knobs for named devices and custom viewports, theme preference, wait strategy, cleanup, hidden selectors, headers, cookies and scripted interactions. With those, a workflow states which version of the page it wants to keep and doesn't settle for whatever loaded after navigation.

Full-page capture has its own trap. Height can keep changing while lazy content, web fonts and images arrive, so a browser may hand back an image whose bottom is still blank. The remedy is procedural: scroll step by step, wait until the document height stops moving, jump back to the top, take the shot, and then check the width and height of the finished PNG or JPEG against the document. If the page never settles or keeps growing, the call returns a typed error. A visible failure beats incomplete evidence passed off as complete.

Cutting tall pages into readable regions

Squeeze a 12,000-pixel screenshot into a small vision-model input and the text turns to mush. The service's answer is a regions flag on the request.

curl --request POST https://api.scrinly.com/render/screenshot \
  --header "Authorization: Bearer $SCRINLY_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "url": "https://example.com",
    "device": "macbook-pro-14",
    "theme": "dark",
    "fullPage": true,
    "regions": true,
    "format": "jpg",
    "quality": 80
  }'

The reply carries the whole screenshot plus ordered crops for the navigation bar, hero, body content, calls to action, footer and any other detected areas. A manifest lists each region's bounds, order, confidence, coverage, dimensions and hash. Every crop is cut from the final delivered screenshot, never from a second capture that might differ.

A design flag adds deterministic evidence about colors, typography, spacing, layout and motion, sanitized and bounded in size. No AI model takes part in producing it.

When region processing breaks after the screenshot itself succeeded, the caller still gets the screenshot, and only the surcharge for regions is refunded.

Evidence that can't be overwritten

Two captures of one URL must never clobber each other. Generated files are therefore stored under names derived from a hash of the final encoded bytes.

{storagePrefix}/{screenshotSha256}.jpg

Different bytes produce different keys, and identical captures can safely share one. Results point straight at the storage provider rather than at an image proxy. Default storage sits on Backblaze B2, and REST API customers can route assets into their own B2, S3 or R2 buckets instead. Caching is opt-in through a cache flag, and public screenshot batches can handle up to 100 stored captures, with billing and refunds worked out per item.

Comparing releases without false positives

Two comparison modes exist: exact pixel matching, and a perceptual mode that tolerates noise. The output can be an annotated overlay, a contextual heatmap, a binary mask, or metadata alone.

Movement is what makes comparisons hard. Insert a new section under the hero and an ordinary whole-image diff flags everything below it as changed. When both captures have compatible region manifests, though, each logical section is compared separately. The inserted block shows up as added, and a footer that merely shifted down is reported as unchanged.

Monitoring reuses the same pipeline. A monitor grabs a new capture and compares it with a rolling or fixed baseline. When a threshold is crossed, or a region appears or disappears, it opens a visual incident, sends signed webhook or verified email alerts, and reports recovery once the page returns to normal.

Slow screenshot and diff jobs can run asynchronously. The API hands back an account-scoped job id, status polling costs nothing, and a signed webhook can deliver the finished result.

AI only after the deterministic steps

Screenshots, regions, design evidence, hashes and diffs are all deterministic. An optional visual style guide adds model interpretation, but only once that evidence is in hand.

The model, which the server controls, receives a bounded overview image, the ordered region images and a sanitized projection. The target URL, hostname, HTML, cookies, headers, selectors, credentials and any supplied page text are withheld as text. Copy and branding that are visible on screen can still appear in the pixels, of course.

The output covers visual foundation, color, typography, spacing, components, motion, responsive behavior and gaps in the evidence, with claims checked against that evidence. Style guides are cached automatically, and a regenerate flag requests a fresh one. Callers may also pass a temporary OpenAI key in a dedicated header, which isn't stored. If generation fails, the screenshot and regions stay available and only the AI increment is refunded.

Exposing it to agents over MCP

The remote MCP server offers four tools:

• capture_screenshot
• compare_screenshots
• get_job_status
• get_usage

Connection works through OAuth, with an API-key fallback. In Codex it looks like this:

codex mcp add scrinly --url https://mcp.scrinly.com/mcp

An open-source agent skill complements the server. MCP makes the tools callable, while the skill teaches compatible agents when a capture is warranted, how to choose the smallest credit ceiling that works, what to do while an accepted job is still running so the paid request isn't sent twice, and how to store matching screenshots, regions and hashes as a set.

npx skills add davmixcool/skills --skill scrinly -g

Paid MCP operations require a maxCredits value. If the worst-case charge would exceed the limit the user declared, the request is turned down before any rendering or billing happens. That draws a practical line between what an agent may do alone and what the user pays for.

Design lessons that carry over

1. A successful browser call doesn't prove a complete screenshot.
2. Derived crops should come from the exact image that was delivered.
3. Stable evidence calls for content-addressed storage.
4. Deterministic processing should run before AI interpretation.
5. Partial success should return the useful work and refund only the part that failed.
6. Autonomous tools need explicit spending limits as well as authentication.
7. Playwright controls the browser, yet a production visual-evidence workflow needs a good deal of infrastructure around it.

On pricing, the free tier includes 100 credits per month. Spending is one credit for a bare screenshot, three when regions are added, one for each visual diff, and up to six for a newly generated, platform-funded style guide.