Playwright and TypeScript Without Knowing TypeScript: An Engineer, an AI Layer and Four Gates

A repository that can be cloned, with Skills and Rules inside and an MCP-driven Playwright that walks through the application unaided, is easy to promise and slow to deliver. This one exists as a working project: clone it, run a single command and the tests execute against a live site. The story behind it starts with a legacy suite and a question about languages.

The migration that never got approved

A QA team had spent four years dragging a legacy Selenium suite forward, and some colleagues who later moved into development had known it even longer. Its core was thousands of C# UI tests, accumulated across the years by authors who had mostly left the company. Moving to Playwright had been on the wish list for a long time. It is faster, steadier and nicer to work with, and still the move never happened.

The reason was mundane. Nobody had a hundred spare engineering hours to pour into a rewrite whose business value is nil, and a manager cannot sign off on "the new tool feels better". Technical arguments are always available, yet the bigger the suite, the pricier the migration and the thinner the gains appear beside it. The rewrite has to be paid for upfront and in full. Its return never shows up as a figure in a report: it arrives as fewer failing tests and quicker fixes spread over a couple of years. The idea therefore stayed parked. Playwright and TypeScript kept appearing in more and more job ads, and doing nothing kept getting more expensive, for the project and for the person.

Then another team started something new, a clean slate: no legacy, no wall of tests that must not break and no migration to justify. An offer to build UI and API automation there brought Playwright and TypeScript into daily work at last. It also exposed an uncomfortable fact. Java and C# were familiar and ten years of test code stood behind them, but this was testing and not development, which is a different depth. JavaScript, TypeScript and Playwright itself were new. A question surfaced that would have sounded rhetorical two years ago: how much does not knowing the language hold anyone back in 2026, with AI around?

Where the experiment came from

No single day triggered the move. It built up as the sense that merely testing and writing tests leads nowhere. QA earns its keep not through bug counts or lines of code but through shaping product quality via processes, documentation, automation and communication, and over the years a systematic approach has beaten individual heroics.

Two things pushed in the same direction.

One push came from a crypto trading bot written in Python with Claude Code by someone who knew no Python. The run began with the thrill of a first success and ended in collapse once the code outgrew anyone's understanding, and no one could say why tests failed or how to repair them. The lesson was the gap between producing code and building a system: the bot worked, yet it could not be maintained. No map of the project existed, fixes bred new problems in pairs, and tokens vanished into rounds of describing symptoms instead of finding causes.

The other push came from daily work, where AI was creeping into QA routines. Trials covered a pipeline running from task to finished test, phrasing rules so the model follows them to the letter, and making the whole flow repeatable and controllable. Colleagues used AI as a smarter search box or a code vending machine. Hardly anyone wrapped it in a system that would make it predictable, which looked like a wasted chance, since the systematic part holds the value.

Those two threads produced the Playwright project, framed as a hypothesis: how far can an engineer with strong fundamentals get with AI doing the typing, and is a complete test framework feasible without writing a line in a language the engineer cannot read fluently?

Why TypeScript and not C# or Java

The first check was the obvious one. Playwright exists for C# and Java as well, so perhaps switching languages was unnecessary. The difference turned out not to lie in driving the browser. That part is identical everywhere, with the same API, the same selectors and the same browsers. The difference lies in how tests are organised and run.

In JavaScript and TypeScript, Playwright comes with a native test runner covering parallel execution, fixtures, reports and automatic tracing. For C# and Java there is none. Teams reach for whatever the ecosystem already uses, whether xUnit, NUnit, JUnit or TestNG, and Playwright merely supplies base classes for them, so organisation and parallelism are hand-built. That is the actual constraint: not strange syntax or a missing method, but rebuilding the wheel, including parallelism, per-thread data isolation, plumbing and reporting, all problems already solved once and best avoided. Next to it stood an option that arrived fully assembled, in a language nobody knew. Playwright grew up beside JavaScript, so there it is a native and not a visitor.

What "built without knowledge" really means

Calling the project built without knowledge would be more than inaccurate. It would be an outright lie. Ten years of QA experience stand behind it: building processes from scratch and testing systems of every kind, from web applications and APIs to mobile apps. That baggage is what kept the work from drowning in a sea of generated code. TypeScript and Playwright syntax stayed a dark horse throughout.

A few courses helped. Between earlier experiments and this build, several Anthropic courses dealt with designing agents, chaining skills and wording tasks for AI. They complemented experience without replacing it. What counted was the posture. The project did not come from a vibe coder asking for some tests. It came from an engineer who told the model the rules, the architectural limits and what needed checking, and then let it act inside those walls, which turned a messy exchange into a managed process.

A skeleton, with two caveats

Two points need stating before the structure and its decisions.

First, the project is a skeleton and not a product, to be taken, adapted and broken. It is shared to show an approach, not to claim perfection. Someone after a ready-made tool will be disappointed, while someone after a model for building a system around AI will find one. Teams have their own constraints, security demands and integrations, and a design that suits a public demo can need real rework behind corporate SSO, MFA, special test-data rules and a test management system. So the reasoning behind the code is the portable part.

Second, the toolset was minimal. It consisted of Claude Code on the basic twenty-dollar monthly subscription, Visual Studio Code as the editor, a configured MCP server for Playwright that lets the model explore the application on its own and, probably most important, a fresh mind and some free time. No expensive corporate solutions, no complex infrastructure and no distributed test runners were involved. Everything runs on an ordinary laptop, which is part of the message: starting does not require anything supernatural, only an understanding of what is being built and the wish to build it.

What the project covers and how it is laid out

The target is Toolshop, a public demo online store made for practising automation. The choice matters: no closed system, no business context, so anyone can clone the repository, install dependencies with one command and run tests against a live site, with no accounts to create and no documentation to dig through.

The suite has thirty-seven tests, thirty-four passing and three failing deliberately so the report shows what a failure looks like. API tests cover server logic, UI tests cover the interface, and hybrids create state through the API before checking it in the browser. That is plenty to demonstrate the architecture and far too little to say anything about product coverage, which depends on a specific product and its business risks and not on how many tests a demo contains. The directory layout:

config/
 env.ts // The only place where environment variables are read
 timeouts.ts // Timeouts, also in one place
core/
 run-user.ts // The user registered by this specific run
 global-setup.ts // Register → UI login → data snapshot → download OpenAPI spec
 global-teardown.ts // Cleanup after the run: remove favorites and the user
 allure-labels.ts // Parses [Section / Subsection] from the test name into report structure
 allure-categories.ts // Buckets for triaging failures in the report
api/
 generated/schema.d.ts // Types generated from OpenAPI, never touched by hand
 types.ts // Domain types derived from the schema
 client.ts // openapi-fetch, paths and params checked by the compiler
 contract.ts // Runtime validation of API responses against the schema
 auth.api.ts // Typed clients on top of api/client.ts
pages/
 base.page.ts // Base page class
 components/ // Elements present on every page (site header)
 auth/ // Page Objects for authorization pages
 account/ // Account area: account, profile, favorites
 shop/ // Catalog, product page, checkout
fixtures/
 base.fixture.ts // All dependency injection, test/expect only imported from here
data/ // Strings and routes, so they don’t live inside tests
utils/ // Logger, reference data snapshot
tests/
 auth/ // Everything auth-related in one folder
   auth.api.spec.ts // API: login, token lifetime
   registration.api.spec.ts
   login.ui.spec.ts // UI: login form, errors, logout
 catalog/
   products.api.spec.ts
   catalog.ui.spec.ts
 cart/
   cart.ui.spec.ts
 favorites/
   favorites.api.spec.ts
   favorites.hybrid.spec.ts // Hybrid: state via API, verification via UI
 profile/
   profile.hybrid.spec.ts
 contract/
   contract.api.spec.ts // Matching live API responses against the spec
 demo/ // Tests tagged @demo that fail on purpose
docs/ // Feature maps, checklists, test cases, plans
.claude/ // AI layer: rules, skills, agents, hooks```

Structure decisions that scale

• Feature first, layer second. One choice looks odd at first yet decides how well everything scales: tests are grouped by feature, and the layer is a filename suffix, as in login.ui.spec.ts, auth.api.spec.ts and checkout.hybrid.spec.ts. A developer changing authorisation logic finds every related test under tests/auth/ and does not trawl three folders. With thirty tests nobody notices, and a layer-based layout looks neat. With three hundred, tests/ui/ becomes a junk drawer of sixty files, because a changed feature changes every test it owns, whatever the layer. Scattered tests turn analysis into searching.
• When even features get crowded. With thousands of tests a feature folder can swell to hundreds of files. The next move is subfolders by subfunction or scenario, such as tests/auth/login/, tests/auth/registration/ and tests/auth/password-reset/, which extends the same idea of keeping context beside the code. Equally, hundreds of tests should not share a single file, since that makes an unreadable, unreviewable monster.
• Hybrid as the starting point. The demo splits into API, UI and hybrid tests, but a real project need not mirror it or produce three layers per feature. There may be no standalone API worth testing, or UI tests may already be hybrids, with preconditions set up through the API or database and only the interface-relevant part checked in the browser. That is legitimate, because separation for its own sake is a mistake. Writing hybrid UI tests from day one avoids pure UI tests that haul state through the browser and burn minutes. Structure is a means, and each additional abstraction adds upkeep.

Four decisions that explain the rest

The repository records fourteen architectural decisions in .claude/rules/architecture.md, and each answers "why this way" and not "what is this". Four of them are retold here, since the rest are better left in the repository than paraphrased.

Every run registers its own user

The usual setup is one test account shared by every test. That holds until the suite grows and tests run in parallel. Then one test empties the cart just as another checks that a product is in it, producing a failure that appears once in twenty runs and never on the debugger's own machine.

The rule sounds dull: every run gets its own sandbox. That is different from promising to avoid shared data or to tidy up afterwards. It means there is physically nobody to collide with. On the public demo, a fresh user registered in global-setup does the job. A corporate environment may block that through tenant limits, shared entities or a data-recreation job already in place. The principle survives and the method of creating the sandbox is a separate question.

The same reasoning gives retries: 0. A retry works as a silencer and not as insurance: it paints a real race condition green and defers the bill to the day it hurts most. A test that needs a retry is not flaky but is reporting an isolation fault, and that report deserves a hearing. The cost is stated plainly: on the demo, registering a user takes one request and always succeeds, and elsewhere the mechanism will differ.

Types come from the spec, and a contract test checks the spec

These are two different checks that get confused constantly.

• The first is compile-time. The API has an OpenAPI document and types are generated from it, so a field referenced in code but absent from the spec stops the project from compiling. It takes two seconds and needs no request to the server.
• The second asks whether the server really matches its own spec. A generated type cannot prove that, since it only proves agreement with the document. Contract tests hit the live API and compare real responses against the schema. On the very first run they found a path mismatch: the document said products/{productId} while the code used products/{id}. Another time they showed that the catalogue filter was calling by_category_slug, a parameter that is not in the spec at all. The server accepts it and the tests are green, so everything "works". But a parameter outside the contract cannot diverge from the contract. It can only vanish quietly one day.

A contract test is therefore not a check on whether an endpoint works. It checks whether this is the endpoint everyone agreed on.

A token that lives for three hundred seconds

The obvious design logs in once before the run, saves the token to a file and lets every worker read it. It nearly went that way. But this API's token lives exactly three hundred seconds, which a dedicated test verifies by comparing two fields inside the token, and the suite takes longer than that. The slowest test would arrive with a dead token and fail with a 401, which looks like a permissions fault and not an expiry, so debugging would start in the wrong place.

So the file keeps credentials and not tokens. Every worker holds its own token in memory and renews it early. A consequence follows: always fetch the token exactly where it is used, and never lift it into a variable at the top of a block. A single such line revives the very problem the design set out to remove.

Snapshot reference data once and mutable state never

Categories, brands and the product page do not change between runs. They can be fetched once before the start and handed to tests, so thirty tests do not each request the same list. Favourites, the cart and the profile must never enter such a snapshot, because the tests themselves change them. Snapshotting mutable state is not an optimisation. It produces data that lies with a delay: a test reads what was true five seconds earlier, fails, and someone hunts a bug in an application that does not have it.

Cleanup obeys the same logic. It runs in order, entities first and their owner afterwards, and it never fails; short permissions only produce a warning. A failing teardown turns a single red test into a red run with the wrong explanation, the worst kind of red.

The AI layer

Consider what happens when Claude Code opens in a project folder and gets the request "write tests for the cart". The model has the request and nothing more. It does not know that shared accounts are forbidden, that locators must come from a verified map, or that asserting on a global counter in a parallel suite is a trap. It guesses, and guesses reasonably well, so the code looks decent and probably runs. Most people stop at that point and call it working with AI.

The AI layer rephrases the problem. The question stops being what the model will guess and becomes what it must know and what it may never do. Five components make it up, each with its own job.

• CLAUDE.md, always loaded. A file in the project root that loads at the start of every session. It works as onboarding for a new colleague, in this case the model, squeezed into two screens: what the project is, where things sit, the five most important points and what is banned outright. The word "always" sets the budget, because each conversation pays for the file in context. Only what is needed every time goes in. Forty style conventions belong in a separate file that it links to.
• rules/, reasons and not commands. These files hold architecture decisions, code conventions and browser procedures. An instruction says not to hoist the token into a variable. A rule explains that the token lives three hundred seconds, so one lifted to the top dies mid-run and fails with a 401 in the slowest test. With the reason in hand, the model needs no specific prohibition: it grasps the class of problem and spots it in unfamiliar surroundings.
• skills/, procedures with a gate. A skill is a playbook for one job: steps, a description, trigger phrases, a designated model and a list of permitted tools. The tool list is a real limit and not documentation. The skill that writes test cases cannot execute commands, so it cannot start running tests just in case, however keen it is.
• agents/, a separate head with fewer rights. Two practical reasons justify them. Context is one: a page dump with every attribute is thousands of tokens of junk, and if it lands in the main conversation, markup crowds out the checklist discussion. An agent works in its own context and returns a result and not raw material. Boundaries are the other. The research agent has no access to test code, so reconnaissance yields documentation and not code as a physical fact. The agent that writes tests may not run the suite, so writing without running holds by itself.
• hooks/, enforced and not remembered. A prose rule lasts as long as someone remembers it, so the most expensive bans become hooks: small scripts that fire before an action and can veto it. Generated types and .env are uneditable. git push --force, --no-verify and --amend are rejected automatically. A hook neither persuades nor explains afterwards. The action does not happen, and the reply names the reason and the proper route.

In one line each: CLAUDE.md is the day-one reading for a newcomer, rules are team standards, skills are process playbooks, agents are interns with limited access, and hooks are the turnstile that stays shut.

The same layer on a live ticket

Abstractions make sense only once a real task passes through them. Take PROJ-431, "pay for an order by card". The normal order of work has nothing to do with AI: someone tries the feature by hand, someone writes a checklist, someone converts it into cases, someone automates them. One person may do all four or three people may split them. For the pipeline it does not matter, which is the point, because the pipeline lays itself over the existing process instead of replacing it. The task number is the thread running through everything, preferred over feature or branch names because it outlasts renames and reassignment.

Step one: the checklist

The QA engineer starts the skill and hands it the number. With MCP linked to Jira, the skill reads the ticket directly, covering the description, the acceptance criteria and, frequently most useful, the comments where a developer explained half the behaviour without updating the description. With no MCP or an empty ticket, the engineer describes the feature in the prompt. The pipeline needs an input and does not care where it comes from.

Then the step everything was built for begins. The skill does not derive a checklist from the ticket text. It summons the research agent, which enters the application, local or test environment, visits the feature's pages, clicks elements, reads the live DOM, calls the API on its own and compares responses with the spec. Only after that is the checklist written, from what actually exists and not from what the ticket promised. Acceptance criteria are inputs, not truth, and half of their mismatches with reality are the task's first bugs, found before any code exists.

Two files come out, and keeping them apart is the lesson that matters most here. The first is the checklist: prioritised business scenarios in the team's format and not element checks. A red border on a field is no scenario. A user with someone else's card being stopped and told why is one. The second is the feature map, holding verified locators, API fields, validation rules and gotchas found on the way. The checklist is the task's direct result and the map is stock for later steps. The QA engineer reads the checklist, strikes what is unnecessary, adds what only a person knows, and that review is a gate: nothing moves on without sign-off.

Step two: test cases

Whoever did step one may continue, or a different person if manual testing and automation are split. A second skill starts with the same task number and locates the checklist and map by itself. The engineer picks what to cover, all of it or only what was agreed. The skill turns those items into cases and settles one thing per case: the layer the check belongs to.

• Rules enforced by the server become API tests.
• Things the user sees become UI tests.
• A check needing fifteen clicks of setup for one assertion becomes a hybrid, with state built through the API and only the visible part checked by eye.

Errors here cost more than poorly written cases, because they breed slow, overlapping coverage that nobody later dares to delete. The skill also attaches a service section to each case: data to use, locators, what must be unique, what is cleaned up and under which account. To a person it looks like paperwork. To the next step it is a ready spec. The engineer reviews and approves, which is the second gate. Teams on Qase or TestRail can add a skill here that uploads approved cases via API and later marks them automated.

Step three: automation

The automation skill also takes the task number and fetches the checklist, cases and map itself. Its first act is unexpected: it checks how fresh the map is. Every section carries a verification date, and anything older than a month stops counting as fact and becomes a claim. The skill neither refuses nor trusts blindly. It moves stale parts into a risks section of the plan and passes the decision to a human, either re-run reconnaissance or accept the risk. A stale locator will surface regardless, but three steps later as a timeout whose cause is hidden.

Next comes the plan: which files appear, what data each test uses, what is created before the run and removed after it, which page and client methods are added and what questions remain. The plan is the third and weightiest gate, because it decides whether the batch is safe to run in parallel, and changing that afterwards is costly.

Once approved, the plan goes to the code-writing agent, whose defining trait is that it does not invent locators. If a needed one is missing from the map, it neither guesses a similar one nor hides the gap by skipping the test. It leaves a note in the code that nobody verified the locator and reports it. The test is still written and fails, which is right, because the problem stays visible and does not dissolve.

What used to be separate commands now lives inside this step: run, read the error, fix. Each test gets two attempts, then the agent stops and admits defeat. One sequential run checks the logic and two parallel runs check isolation. No extra button is needed, since none of this calls for a human decision. It is a procedure, kept in the rules.

The prohibitions here stand apart and are not up for debate:

• Do not disable a failing test to turn the run green.
• Do not raise retries.
• Do not quietly weaken a check to match observed behaviour.

If the application behaves wrongly, a red test is the correct result.

Step four: review and commit

Review and commit belong to two skills and not one, a deliberate split, since a step that starts with the intention to commit is worse at finding reasons not to. The review carries project-specific knowledge no generic reviewer has: what a shared entity means here, what a token at the top of a block puts at risk, why asserting on a global counter misleads. Each entry on its list comes from a past breakage of this suite. To stop the split from becoming a formality, the review stamps a fingerprint of the diff it examined, and the commit refuses to proceed if the diff has changed since. That is not a lock. It is a question with an exact answer: was this code reviewed, or different code? The difference between a process and a habit sits right there.

When the whole chain has run, PROJ-431 holds the full story: what was researched, what was chosen for verification, what is covered by automation, what remains manual and what will not be automated and why. The reason is not a love of documentation. Six months later the question of why a scenario is not covered will come from a living person at a retrospective, and an answer needs to exist.

What outlives the task

The setup's main risk is not bad generated code. It is the context decaying into a dump six months on, with nobody sure which file to trust. So artefacts get three separate lifespans, and mixing them is forbidden.

• Rules last as long as the project and change only alongside the code.
• The feature map lasts for years and belongs to the feature and not the ticket. It is extended with dated sections, new pages and fresh gotchas, never rebuilt per task.
• Checklists, cases and plans belong to one task and turn into history when it closes.

Without that split, a year later the locator map exists in ten copies scattered across task numbers, half of them wrong and no way to tell which half. When one feature's map swells, as large features' maps do, the cure resembles the one for code: cut it into pieces along natural seams such as page, subfeature or scenario, and keep a short index on top of what lives where. A hard input rule applies as well: only verified facts enter the map, meaning what the agent actually saw in the live application, with a date. Guesses and ticket retellings do not belong. A map that accepts them is garbage within a sprint, and facts cannot be sorted from guesses afterwards.

None of it is a universal recipe. Four pipeline steps, five AI-layer parts, gates and artefact lifespans are one possible arrangement. Teams vary in headcount, product and tooling, so half the details will land differently. Some will work, some will need adapting and some will not stick, and that is acceptable.

The central point holds regardless. Build the AI layer properly, so the model follows the team's rules inside the team's boundaries using the team's artefacts, and routine work leaves the team's plate: reading tickets, writing similar cases, picking layers, hunting locators, rerunning and repairing failures, reviewing against the same pitfalls for the hundredth time. QA engineers are not replaced and the existing process is not cancelled. Time simply stops going into what can be handed off.

Are seniors still needed?

Senior engineers are needed more than ever, with a changed role. David Heinemeier Hansson has observed that the best agent-driven acceleration he has seen came from the most senior people, the ones able to judge whether the agent's output is fit for use.

Engineers were once valued for writing code, knowing syntax and delivering tasks fast, which made sense while code was the main product. With AI that value has shifted by an order of magnitude and not a few percent. The machine writes the code, and the human decides what to write, why, inside which limits and how to verify that the result is not lying.

Picture two specialists. The first opens Claude Code and asks for a login test. It gets written and passes, and the specialist feels productive, which is true for today. A month later the test breaks, the cause is a mystery, patches go in at random while tokens burn, and eventually everything is rewritten or the test is deleted and forgotten.

The second specialist designs first: how tests are isolated, which pattern the API client follows, how tokens are managed and what happens to state that shifts between tests. Only then does the instruction go to the model: write a login test under these rules.

The first holds a test that works now. The second holds a system that keeps working, because it stands on architecture and not luck. On a small set the gap is invisible, except that the first finishes ten or fifteen tests in ten minutes while the second needs two hours or more for the foundation. After a hundred, two hundred or a thousand tests and fifty to a hundred pipeline runs, the gap is obvious.

Does that mean a programming language must be learned?

Yes, though not for writing code. A language is needed to understand what the AI produced, to catch mistakes in generated code, to review pull requests and to decide what reaches the main branch. Syntax is a tool. A complex construct from memory is out of reach for a tester in unfamiliar territory, but architecture is known ground: what the system should look like, how tests stay isolated, how pipelines are organised and what a good report contains. That foundation made the Playwright project possible.

Architecture does not come from a prompt. AI can propose it, sometimes well, but architecture is a choice among answers and not just an answer. A model offers five options that all look reasonable. Which one will fit in six months, with ten times the tests and altered requirements, is not something that gets generated. It comes from experience, from the calluses earned fixing flaky tests, working out why a test passes locally and fails in CI, and discovering that a shared account is hidden coupling waiting to explode.

Companies have no use for people who only press buttons and write prompts in Claude Code or Cursor. Such people are plentiful and will be more so, and the market is full of vibe coders who believe they struck gold, which holds only briefly. What is wanted are people who understand why the AI did something, notice when it is wrong, frame tasks so the model returns a working solution and not a polished mistake, and build a system instead of feeding tokens into endless "try again, not like that" rounds.

Process, automation and pipeline experience is no archive. It is the foundation under competent AI work and separates the person who makes chaos from the one who makes a system. Knowledge once served to do things by hand. Now it serves to set tasks for AI and to check the result. The role changed and the value of expertise did not. It is arguably easier to see, since beside AI it is clear who merely generates and who understands what was generated and why.

What the experiment shows

A framework was built in a language its builder does not know, and this is not a story about brilliance. It is a story about not trying to guess what AI would do, and spending the time to explain the rules of the game instead: what is allowed, what is not and why this way and not that way. It is boring, it has nothing to do with vibes and it takes time before the first result appears. That is exactly what works.

The repository does not prove engineers are obsolete or that languages stopped mattering. It is a live test of how the engineer's role shifts when value moves from writing code to framing tasks, designing systems and checking results. Writing code by hand remains a skill. A second one has joined it and proved just as important: understanding what was generated, why, and what comes next.

AI does not replace the engineer. It becomes the quickest and most obedient junior imaginable, tireless, never arguing and always ready for another attempt. It has one quirk: it delivers a correct solution and utter nonsense with the same confidence, and only someone who understands the work can tell them apart. So the engineer must stay at the wheel as the one who knows the destination and the road, not as someone merely holding the steering wheel.