Growing a Playwright Script into a Tool: Storage, Retries, CLI and AI Guardrails

Many repetitive browser chores follow the same loop: open a site, navigate, type something, read the result, save it, repeat. Each step is trivial, but doing them by hand every day adds up. This walkthrough takes that loop from a ten-line Playwright script to a small Node.js tool with persistent history, duplicate detection, retries, a command-line interface, central configuration and a guarded natural-language front end.

The guiding flow is: a request comes in, a JavaScript workflow runs it, a real browser does the work, structured results come out and land in a database or report. A related piece covers building a modular JavaScript automation toolkit from scratch; here the emphasis is on the decisions that make such a tool dependable as it grows.

Step one: drive a real browser

Playwright lets JavaScript control Chromium, Firefox or WebKit. The starting script launches a visible browser, loads a page, waits until the DOM is ready, prints the title and shuts down.

const { chromium } = require("playwright");

async function openWebsite(url) {
    const browser = await chromium.launch({
        headless: false
    });

    const page = await browser.newPage();

    await page.goto(url, {
        waitUntil: "domcontentloaded"
    });

    console.log("Title:", await page.title());

    await browser.close();
}

openWebsite("https://example.com");

headless: false lets you watch the automation while developing, and waitUntil: "domcontentloaded" resolves once the HTML is parsed instead of waiting for every image and script. Everything that follows is structure built around these capabilities.

Wrap raw actions in named functions

Scattering selector code across a project makes every page change expensive. Instead, small functions own one interaction each, and a higher-level function combines them.

async function fillSearchBox(page, query) {
    const searchBox = page.getByRole("textbox");

    await searchBox.fill(query);
}

async function clickSearch(page) {
    await page.getByRole("button", {
        name: "Search"
    }).click();
}

async function performSearch(page, query) {
    await fillSearchBox(page, query);
    await clickSearch(page);
}

The locators use getByRole, which finds elements by their accessible role and name. It tends to survive markup refactors better than class selectors. Callers now express intent without knowing the mechanics:

await performSearch(
    page,
    "JavaScript automation"
);

If the search form changes, only fillSearchBox and clickSearch need updating.

Turn pages into structured data

Raw HTML is hard to work with. The next function selects every .article element and, inside the page context, maps each one to a plain object with a title and author.

async function extractArticles(page) {
    return await page
        .locator(".article")
        .evaluateAll(elements => {
            return elements.map(element => {
                const title = element
                    .querySelector(".title")
                    ?.textContent
                    ?.trim();

                const author = element
                    .querySelector(".author")
                    ?.textContent
                    ?.trim();

                return {
                    title,
                    author
                };
            });
        });
}

evaluateAll runs the callback in the browser against all matched elements and returns the serialized result to Node. The optional chaining (?.) keeps a missing .title or .author from throwing; that field is simply undefined. Using it is then a single call:

const articles = await extractArticles(page);

console.log(articles);

Once the page is an array of objects, you can filter, compare, store or pass it along like any other data.

Give the tool a memory with SQLite

Results that vanish when the process exits are of limited use. The sqlite3 package provides a file-based database, and the setup below creates a results table on first run, with an auto-incrementing ID and a timestamp default.

const sqlite3 = require("sqlite3").verbose();

const db = new sqlite3.Database(
    "automation.db"
);

db.run(`
    CREATE TABLE IF NOT EXISTS results (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        title TEXT,
        author TEXT,
        source TEXT,
        created_at DATETIME
            DEFAULT CURRENT_TIMESTAMP
    )
`);

Saving a row goes through a helper that wraps the callback-based db.run in a promise so it works with await. The ? placeholders bind values safely, so scraped text containing quotes cannot break or inject into the SQL.

function saveResult(result, source) {
    return new Promise((resolve, reject) => {
        db.run(
            `
            INSERT INTO results
            (title, author, source)
            VALUES (?, ?, ?)
            `,
            [
                result.title,
                result.author,
                source
            ],
            error => {
                if (error) {
                    reject(error);
                    return;
                }

                resolve();
            }
        );
    });
}

History enables change tracking, run comparisons, reports and an activity log.

Avoid storing the same result twice

Repeated runs will often see the same items. A lookup helper checks whether a result with the same title already exists for a given source.

function resultExists(result, source) {
    return new Promise((resolve, reject) => {
        db.get(
            `
            SELECT id
            FROM results
            WHERE title = ?
              AND source = ?
            LIMIT 1
            `,
            [
                result.title,
                source
            ],
            (error, row) => {
                if (error) {
                    reject(error);
                    return;
                }

                resolve(Boolean(row));
            }
        );
    });
}

The workflow then only inserts results that are new:

for (const result of articles) {
    const exists = await resultExists(
        result,
        "example.com"
    );

    if (!exists) {
        await saveResult(
            result,
            "example.com"
        );
    }
}

This check-then-insert approach is fine for a single sequential process. If several runs could write at the same time, two of them may both see "not found" and insert duplicates. A sturdier option is to add a UNIQUE constraint on (title, source) and use INSERT OR IGNORE, which lets the database enforce the rule atomically.

Make flaky steps retryable

Real automation is less predictable than a demo. Networks drop, pages load slowly, servers answer differently, and elements are not always present right away. A generic retry helper runs an async operation up to a given number of times, waits between attempts and rethrows the last error if all of them fail.

async function retry(
    operation,
    attempts = 3,
    delay = 1500
) {
    let lastError;

    for (
        let attempt = 1;
        attempt <= attempts;
        attempt++
    ) {
        try {
            return await operation();
        } catch (error) {
            lastError = error;

            console.log(
                `Attempt ${attempt} failed`
            );

            if (attempt < attempts) {
                await new Promise(resolve => {
                    setTimeout(
                        resolve,
                        delay
                    );
                });
            }
        }
    }

    throw lastError;
}

Wrapping a navigation in it looks like this:

await retry(
    async () => {
        await page.goto(
            "https://example.com",
            {
                waitUntil:
                    "domcontentloaded"
            }
        );
    },
    3
);

Two things are worth noticing. The delay is fixed; for services that may be overloaded, an increasing delay (exponential backoff) is usually kinder. And retries only make sense for operations that are safe to repeat: reloading a page is fine, while resubmitting a purchase form is not.

Bundle the workflow in a class

With several helpers in play, a single object can own the page and expose the steps. WebAutomation receives a browser, opens a page in start(), navigates through retry, performs the search and closes the page.

class WebAutomation {

    constructor(browser) {
        this.browser = browser;
        this.page = null;
    }

    async start() {
        this.page =
            await this.browser.newPage();
    }

    async open(url) {
        await retry(async () => {
            await this.page.goto(url, {
                waitUntil:
                    "domcontentloaded"
            });
        });
    }

    async search(query) {
        await this.page
            .getByRole("textbox")
            .fill(query);

        await this.page
            .getByRole("button", {
                name: "Search"
            })
            .click();
    }

    async close() {
        if (this.page) {
            await this.page.close();
        }
    }
}

Application code then reads as a short script of intentions, with the browser lifecycle kept outside so one browser can serve several automation objects.

const browser =
    await chromium.launch({
        headless: true
    });

const automation =
    new WebAutomation(browser);

await automation.start();

await automation.open(
    "https://example.com"
);

await automation.search(
    "JavaScript"
);

await automation.close();

await browser.close();

In production, put the close() calls in a finally block.

Expose it as a command-line tool

Editing code to run a different task does not scale. The Commander library turns the project into a CLI with a name, description, version and subcommands. The visit command below takes a URL argument, opens it headlessly and prints the page title.

const { Command } = require("commander");

const program = new Command();

program
    .name("webtool")
    .description(
        "Browser automation utility"
    )
    .version("1.0.0");

program
    .command("visit")
    .description(
        "Visit a webpage"
    )
    .argument("<url>")
    .action(async url => {
        const browser =
            await chromium.launch({
                headless: true
            });

        const page =
            await browser.newPage();

        await page.goto(url);

        console.log(
            await page.title()
        );

        await browser.close();
    });

program.parseAsync();

parseAsync() is used instead of parse() because the action handler is asynchronous. The tool is then run from a terminal:

node webtool.js visit https://example.com

At this point the script has become a tool others, or a cron job, can call.

Centralize configuration

Hard-coded values spread quickly. A single configuration object groups browser, database and output settings.

const config = {
    browser: {
        headless: true,
        timeout: 30000
    },

    database: {
        file: "automation.db"
    },

    output: {
        directory: "results"
    }
};

Launch code then reads from it, including a default timeout applied to every Playwright action on the page:

const browser =
    await chromium.launch({
        headless:
            config.browser.headless
    });

const page =
    await browser.newPage();

page.setDefaultTimeout(
    config.browser.timeout
);

Keeping settings in one place makes the jump from experiment to maintainable application easier.

Put a validated AI layer in front

A conventional tool exposes fixed commands:

search
open
extract
save
report

A language model can instead interpret a request such as asking for recent JavaScript articles from a site to be found and stored and turn it into a structured task object:

const task = {
    action: "search",
    query: "JavaScript",
    saveResults: true
};

The model must never be allowed to run arbitrary code. Every task it produces is checked against an allow-list of supported actions before anything executes.

const allowedActions = new Set([
    "visit",
    "search",
    "extract",
    "save"
]);

function validateTask(task) {
    if (!allowedActions.has(task.action)) {
        throw new Error(
            "Unsupported action"
        );
    }

    return true;
}

The resulting pipeline keeps interpretation and execution separate:

Natural Language
       ↓
      AI
       ↓
Structured Task
       ↓
Validation
       ↓
JavaScript Workflow
       ↓
Browser / Database

The model decides what the user seems to want; the application decides what it is actually permitted to do. In practice, validation should go beyond the action name and also check parameters, for example that URLs belong to approved domains.

What the tool is really for

Users care that a tedious chore disappeared, not that Playwright is involved. The same engine fits legitimate uses such as QA and automated testing, site monitoring, internal workflows, collecting data from authorized sources, recurring reports, content management and business process automation. Commercially, that can be packaged as setup plus customization and ongoing maintenance, or as a subscription with usage limits and premium features. The code is the engine; the workflow is the product.

The resulting architecture

Put together, the layers look like this. Cheerio appears alongside Playwright as a lighter option for parsing static HTML where a full browser is unnecessary.

USER
                      │
                      ▼
              CLI / AI REQUEST
                      │
                      ▼
                TASK VALIDATOR
                      │
                      ▼
               WORKFLOW ENGINE
                 │          │
                 ▼          ▼
             PLAYWRIGHT   CHEERIO
                 │          │
                 └────┬─────┘
                      ▼
                  PROCESSED
                    DATA
                      │
                      ▼
                   SQLITE
                      │
                      ▼
              REPORT / EXPORT

Wrapping up

The tool was never designed up front. It grew one layer at a time, each answering a concrete pain:

• Named functions around locators contain the cost of site changes.
• Structured extraction plus SQLite turns one-off runs into history.
• Deduplication and retries make repeated, unattended runs trustworthy; prefer database constraints and idempotent operations.
• A CLI and central config let others run it without touching code.
• An AI front end is safe only behind strict validation of every generated task.

Rather than starting from which framework to learn, start from a repetitive problem worth removing and let the architecture follow.