At first glance, Playwright appears to be a godsend for automation QA experts involved in browser test creation and execution. It is fast, excels at dynamic content handling, has a built-in test runner and test generator, and allows for seamless CI/CD integration.

That said, Playwright has a few shortcomings. It supports a limited number of programming languages, displays inadequate legacy browser support, doesn’t see eye to eye with some third-party tools (like test management solutions or reporting systems), is honed primarily for mobile browsers, presents significant problems in test maintenance (concerning test scripts and locators), and has a steep learning curve. Given such downsides, it makes sense to consider viable alternatives to Playwright.

This article offers a list of top Playwright alternatives, compares the pros and cons of various test automation frameworks, and gives tips on choosing the right Playwright alternative for particular use cases and testing needs.

The Top 12 Playwright Alternatives

Testomat.io

This is probably the best Playwright alternative we know of. Why? Because it is a multi-functional test automation tool that enables not only browser testing but in fact all types of QA procedures to boot (usability, portability, scalability, compatibility, performance testing, you name it). It allows for parallel or sequential cross-browser and mobile testing on multiple operating systems and mobile devices (both Android and iOS). The tool integrates with Playwright and its counterparts (for instance, WebDriverIO), CI/CD tools, and third-party apps (like Jira).

What sets Tesomat.io apart from its competitors is its outstanding test case, environment, and artifact management capabilities, as well as real-time analytics and reporting features. Plus, testers can involve non-tech employees in their workflow, enabling them to utilize BDD format and Gherkin syntax support when describing testing scenarios.

Although for novices in cloud-based test management systems, the learning curve may seem quite steep, the modern AI toolset offered by Testomat.io is an excellent alternative to Playwright MCP. What makes it especially attractive is the ability to choose between the basic free version and two commercial ones, with the Professional tier at $30 per month, which suits startups, small, and medium-size businesses perfectly.

Cypress

If you need to quickly test the front-end of web applications or single-page apps, Cypress is a good choice. It is easy to set up, offers automatic waiting for elements (which eliminates the necessity for manual sleep commands), has superb real-time debugging capabilities, and provides built-in support for stubbing and mocking API requests. Moreover, its cy.prompt and Cypress Copilot tools are AI-powered, enabling code generation from plain English descriptions.

On the flipside, you can write tests only in one language (JavaScript). Plus, tests don’t work in multiple browser sessions, and you have to install third-party plugins for XPath, reports, and other crucial features.

Cypress has both free and paid options (the latter are hosted in the cloud, not on the user’s hardware). The cheapest Team plan, allowing for 10,000 tests, costs $75 a month, and the priciest is the Enterprise plan with customized fees.

Selenium

It is an open-source test automation framework that is honed for cross-browser testing of enterprise-size solutions and mobile apps where extensive customization is mission-critical. It consists of three components (IDE, Grid, and WebDriver), which, unlike Cypress or Playwright, play well with a great range of popular programming languages. Plus, it allows for versatile integrations and parallel testing, enjoys extensive browser compatibility, enables headless browser automation, and boasts b community support.

Among the best Selenium’s fortes are Healenium and TestRigor. The first is an ML-driven self-healing test automation library that adapts to changes in web elements in real-time. The second is a cloud-based AI-fueled tool that enables the creation and maintenance of automated tests without any prior knowledge of coding.

Among Selenium’s disadvantages, one should mention the sluggishness of the script-based approach it employs, the need for third-party integrations (for instance, TestNG), expensive maintenance, and problematic test report generation.

CodeceptJS

The most appreciated advantages of this innovative, open-source testing platform are its simple BDD-style syntax, integration with modern front-end frameworks (Angular, Vue, React, and others) and CI/CD tools, high speed, and automated AI-driven creation of page objects with semantic locators, enabling the swift finding of elements in them. Thanks to its consistent APIs across a gamut of helpers, CodeceptJS users can easily switch between testing engines while interactive pauses for debugging and automatic retries remarkably streamline and facilitate the QA pipeline.

In a word, it is a cross-platform, driver-agnostic, and scenario-driven tool with AI-based features (such as self-healing of failing tests) that can be applied in multi-session checks (typically, functional and UI testing) of web and mobile solutions. The AI providers it integrates with encompass Anthropic, OpenAI, Azure OpenAI, and Mixtral (via Groq Cloud). What is more, the CodeceptJS configuration file allows users to configure other providers within the system. If you need consultations concerning the platform’s operation or devising test cases, you can obtain it on community forums or through GitHub issues.

Yet, its versatility and ease of use come with some downsides, namely the immaturity of AI features, less efficiency in handling complex web and native mobile apps, and limited support for certain cloud platforms.

Gauge

It is a lightweight, open-source framework primarily designed for user journey verification and acceptance testing of web apps. Gauge can perform browser automation when coupled with other tools. The pros of Gauge are its readable and foolproof Markdown test specifications, support for multiple programming languages, wide integration capabilities (including automation drivers, CI/CD tools, and version control solutions), and a ramified plugins ecosystem.

Gauge’s demerits are mostly the reverse side of its merits. While broad-range integration is a boon in itself, it spells excessive reliance on third-party drivers, each of which must be configured and managed directly. Likewise, the open-source nature of the tool means that the support typically comes from the community, which can fail to respond to requests in a heartbeat.

Jest

The in-built mocking capability of this Meta-launched framework enables easy cross-browser testing of separate modules, units, and functions within a solution. Besides, it is simple to set up, with its learning curve being rather mild. However, Jest’s free nature may cost you a pretty penny down the line with maintenance and server-related expenditures accumulating over time. Besides, some users claim that large amounts of code and high-capacity loads dramatically slow the system.

WebDriverIO

This is a great alternative to Playwright for QA teams that rely on CI/CD integration-heavy workflows and are looking for WebDriver-based automation. The framework allows testers to conduct cross-browser and mobile testing with high test coverage, thanks to its extensive plugin ecosystem, which offers enhanced automation capabilities. However, it has significant configuration requirements, lackluster reporting, limited language support (mostly JavaScript and TypeScript), and concurrency test execution limitations.

Testcafe

Unlike the previously mentioned tool, this one doesn’t need browser plugins or WebDriver to run the test, because TestCafe does it directly in real browsers. Its best use cases are those that require parallel test execution on real devices without additional dependencies. Yet, with TestCafe, you can write tests only in JavaScript or TypeScript, and you won’t be able to replicate some user interactions with the device (such as clicks and double clicks).

Keploy

It is free for owners of the Apache 2.0 license. Keploy’s key perk is its capability for automated stub and test generation, enabling QA teams to build test cases out of real-life user interactions. It saves testers time and effort they would otherwise spend on creating tests manually. Such a feature, in combination with numerous native integrations and AI-driven automation, allows experts to radically step up test coverage and suits perfectly for API and integration testing routines across various solutions.

Among cons, a steep learning curve and limited support for non-JavaScript-based applications are worth mentioning.
In addition to the mostly free frameworks mentioned above, let’s explore paid alternatives to Playwright with observability features.

Katalon

It is geared toward testing mobile, app, and desktop applications by both experts and non-tech users. Katalon’s user-friendly UI and AI utilization make it a solid tool for keyword-driven testing with fast scripting potential. Outside its specific hardware requirements, Katalon’s main drawback is the price. Its most basic version (Katalon Studio Enterprise) costs $208 a month, with each new functionality increasing the price. Thus, for the Katalon Runtime Engine, you have to fork out $166 a month more, and for Katalon TestCloud – plus $192.

Testim

It is praised for codeless test recording, easy scalability, reusable test steps and groups, drag-and-drop visual test editor, extensive documentation, constantly available customer support, and plenty of AI-driven features (smart locators, help assistant, self-healing capability, and more). The major downside of Testim is the vendor’s obscure pricing policy. They customize plans to align with test coverage and needs, and extend numerous enterprise offerings (Testim Web, Mobile, Copilot, etc.), the price tag of which is declared on demand.

Applitools

Efficiency, speed, seamless integrations with other testing frameworks, advanced collaboration and reporting opportunities, generative test creation, and AI-fueled visual testing are the weighty assets the platform can boast. However, it is rather hard for novices to embrace, subpar in customization, and provides limited manual testing support. You could put up with these shortcomings but for Applitools’ price. Its Starter plan is $969 a month (to say nothing of the custom-priced Enterprise tier), which makes Applitools an upmarket product hardly affordable for small and even medium-size organizations.

Let’s summarize the information about Playwright alternatives.

Top 12 Playwright Alternatives Contrasted

A detailed comparison is more illustrative when presented in a table format.

As you see, there are plenty of browser testing frameworks, which means that selecting among them is a tall order. Perhaps it is better to stay with the classical Playwright?

Reasons to Choose an Alternative over Playwright

To make a wise and informed decision concerning the choice of a Playwright alternative, you should consider project needs that make Playwright a misfit. Opting for another framework makes sense if:

1. You face specific requirements. The need for better mobile testing capabilities or extensive support for legacy systems calls for something other than Playwright.
2. You look for a milder learning curve. Setup and debugging in TestCafe or Cypress are more intuitive and simple to master for greenhorns in the field.
3. Testing speed matters. Some alternatives (like Cypress) enable faster testing than Playwright does.
4. You lack expertise. Testim and Selenium are no-code frameworks accessible to non-tech users.
5. Multiple third-party integrations are vital. Many tools (CodeceptJS, Gauge, Keploy, WebDriverIO, etc.) offer wider integration options and/or a versatile plugin ecosystem.
6. Constant support is non-negotiable. Users of open-source platforms like Playwright can rely only on peer advice and recommendations. Professional 24/7 technical support is provided exclusively by commercial products.

Conclusion

Playwright is a high-end tool employed for automating browser testing across different platforms and browsers. However, other tools can surpass it in terms of the range of programming languages, legacy browser support, simplicity of use, no-code options, and meeting specific project requirements. Ideally, you should opt for a framework that enables comprehensive cross-browser and cross-platform testing, plays well with multiple third-party systems, provides real-time reporting and analytics capabilities, and is free (or at least moderately priced). Testomat.io is an optimal product that ticks all these boxes.

The post Playwright Alternatives: Top 12 Tools for Browser Automation & Testing appeared first on testomat.io.

The Playwright Model Context Protocol fits right in with modern AI automation testing and helps development and QA teams write, manage, and execute automated tests more efficiently while improving test coverage and stability.

In our Playwright MCP tutorial, you will discover more details about the Playwright MCP server, reveal how it works, and learn how to set up Playwright MCP and benefit from integration with the test case management system.

What is Model Context Protocol?

Model Context Protocol (MCP) is an open protocol that standardizes how applications provide context to large language models (LLMs), developed by Anthropic. MCP is like a USB-C port for AI applications. Just as USB-C provides a way to connect your devices to various peripherals and accessories, MCP provides a formal two-way connection of how AI models integrate different data sources, services and external tools. Do it without requiring custom integrations for each one.

MCP enables you to build agents and complex workflows on top of LLMs, connecting your models with the world, while addressing challenges such as security, scalability, and efficiency in AI-powered workflows.

MCP Architecture: How It Works

MCP follows a client-server architecture where an MCP host — an AI application which establishes connections to one or more MCP servers. It integrates AI models with external data sources and tools, including Google Drive, databases, APIs, and more. They, in turn, make their data accessible via Model Context Protocol servers. Each MCP client AI application maintains a dedicated connection with its corresponding MCP server. Every request to the server can provide context to LLMs in real-time, allowing them to maintain context even across multiple systems.

Components of MCP

Let’s define an architecture in detail where:

• Hosts – applications the user interacts with – Claude Desktop, an IDE like Cursor, and custom agents.
• Clients – components that are responsible for requesting and consuming external context from compliant servers. BeeAI, Microsoft Copilot Studio, Claude.ai, Windsurf Editor, and Postman are some of the popular examples of Model Context Protocol clients.
• Servers – these external programs can make their tools, resources, and prompts available to an AI model through a standard API (Application Programming Interface) and convert user requests to server actions.
• Local data sources – the computer’s files, databases, and services to which Model Context Protocol servers have secure access.
• Remote services – external systems which can be accessed over the internet (e.g., through APIs) and connected to.

Most developers will likely find the data layer protocol section to be the most useful. It discusses how MCP servers can provide context to an AI application, regardless of where it runs. MCP servers can execute locally or remotely.

How MCP Client Interacts With MCP Server

1. The MCP client, typically embedded in AI applications, creates a request for specific data or actions.
2. The MCP client sends requests to the Model Context Protocol server when the AI model needs to access exposed data, tools, resources, or prompts.
3. The MCP server gets these requests and sends them to the right external program or data source. Then, it handles the processing to make sure that the right data is retrieved.
4. The MCP server gets the results from the external program once it’s finished. It then safely sends the response back to the Model Context Protocol client for consumption by the AI app.

What is MCP Playwright for Automation Testing?

The Playwright MCP meaning refers to the Playwright Multi-Context Protocol. When Model Context Protocol is combined with the Playwright cross-browser testing tool, it provides browser automation capabilities and utilizes Playwright’s locators to let LLMs or AI agents interact with web pages through structured accessibility snapshots instead of screenshots. 

To put it simply, Playwright, which is known as one of the popular JS testing frameworks, acts as an MCP Client and connects to MCP Servers that expose various tools, services, or data. This setup helps QA teams and developers develop smart test scenarios, which are able to react to dynamic and live data, orchestrate more complex manual and automation workflows, and simulate real-world interactions with the system under test while keeping comprehensive and realistic automation.

  Here is an example Playwright MCP example in action:

In e-commerce, the Model-Context-Protocol server could provide a searchProducts(query) function. When Playwright sends a prompt to check how the product search bar on a website works, the MCP server would return relevant product details as if from a live database.

In this situation, Playwright’s test automation script sends a search request, which is the prompt, to the MCP server, and the Model-Context-Protocol server then runs its searchProducts function, retrieves product information, and sends this data back to Playwright, simulating the search results that a user would see in real time. 

Key Features of MCP Playwright

Playwright MCP comes packed with a variety of powerful features that make it a must-have for today’s automation testing. Get to know these features, and you can make the most out of it:

• Modular communication. The Model-Context-Protocol modular architecture of Playwright lets you use a set of tools, such as test runners, data generators, and smart validators.
• Tool interoperability. When connecting Playwright to more than one Model-Context-Protocol server, which offer specialized tools (for example, visual tools, accessibility checkers, or API fuzzers), it lets you create complicated Playwright-based test scenarios without making your code too big.
• Remote execution. Running tests on remote Model-Context-Protocol servers at the same time speeds things up and makes them more scalable.
• Dynamic tool discovery. At runtime, Playwright’s MCP can ask a Model Context Protocol server what tools and services are available so that users can make test suites that can change and adapt.
• Structured communication. Playwright’s Model Context Protocol and servers communicate using a standardized format (typically JSON), so that data and commands are exchanged without fail.

Playwright MCP AI Workflow

A typical workflow can be divided into the following phases, each with specific objectives:

1. Setup and Initialization. Firstly, you need the server to be installed and configured so that it can receive commands and translate them into browser actions. Only by establishing the necessary connection can you prepare the environment for the AI agent or LLM to interact with web browsers.
2. Capability Discovery. At this step, an AI client (e.g., an LLM or an AI agent) queries the Model-Context-Protocol server at runtime to discover what tools and services are available. Whether it is navigating pages, clicking, typing, or taking snapshots, AI needs to understand the full range of actions it can perform on a web page.
3. Command Generation. Guided by a pre-defined testing scenario, the AI model generates specific commands for the Model-Context-Protocol server in JSON. Then, it translates test requirements into concrete instructions for browser automation, explaining what the browser needs to do.
4. Browser Execution. At this step, the MCP server receives the commands provided by the AI and uses Playwright to execute them in a real web browser (Chromium, Firefox, WebKit). It interacts with the web page by performing actions like navigating to URLs, interacting with UI elements, and capturing page states.
5. Contextual Feedback and Iteration. Once a command has been executed, the Model-Context-Protocol server provides rich contextual feedback to the AI (in the form of accessibility tree snapshots of the page). After that, AI analyzes this feedback to refine its next steps, generate further commands, or validate results to reach the desired goal.

Pros and Cons of Playwright MCP

While Playwright MCP presents several benefits, there are also potential drawbacks to consider. Understanding both can help you make informed decisions about using it.

Pros	Cons
It allows AI models to identify and interact with web elements based on their context and accessibility to reduce test flakiness caused by minor UI changes and improve the self-healing capabilities of tests.	You need to have powerful infrastructure to run an AI client and a Playwright Model Context Protocol server, especially with a visible browser or many connections.
Playwright’s MCP AI agents can uncover edge cases and unexpected behaviors that might be missed by static tests.	AI’s inability to accurately interpret the web page context and generate appropriate actions leads to wrong or failed tests.
Since the AI can “see” and understand the page, teams spend less time manually updating element locators when the UI changes.	Managing complex, multi-step tasks or several AI agents through Playwright MCP can be tricky and takes careful design and debugging to make sure all actions are coordinated throughout a long user journey.
AI clients use detailed MCP information (page accessibility or network requests) to build relevant and smarter test flows.	Dev and QA teams must be well-versed in understanding Playwright, test automation, and how to work with AI models and the Model-Context-Protocol protocol.
It uses modules for AI datatype conversions to process complex transformations between formats efficiently and convert spatial data for web visualization, easing integration into test environments.	If AI models can access live browsers directly through MCP servers without any security measures in place, there is a risk of data theft.
It provides a standardized protocol for communication to be integrated with various AI models and platforms to create a more flexible approach to AI automation testing.	Limited support for legacy systems may hinder integration with older web applications, requiring additional adaptation efforts.

Why Teams Need to Use Playwright with MCP

• Smart test generation. Teams can create test cases automatically from the latest data of the application, which is based on Playwright’s MCP ability to utilize Large Language Models (LLMs) and develop more tests to increase test coverage.
• Remote debugging. Software engineers can attach to the same Playwright instance for debugging to identify and resolve issues in real time, without the need to replicate the testing environment.
• Shared testing environments. QA engineers can execute tests on a shared Playwright instance without the need to set up separate environments for each team member to accelerate the process.
• Live monitoring. Dev teams can monitor ongoing tests in real time, get rapid feedback, and resolve bugs quickly during testing sessions.
• Load testing & performance analysis. Teams can measure certain performance metrics, such as page loading speed during high traffic, server responsiveness under load, memory and CPU usage during smoke testing, and optimise accordingly.
• Distributed and parallel testing. Instead of running separate Playwright instances for each test suite, teams can launch multiple browser instances in order to improve overall testing efficiency and reduce test execution time.
• Adaptive testing based on live data. Teams can get more accurate test results and fine-tune tests using A/B testing and real-time user data.
• Self-maintaining test suites. Teams can forget about ongoing script maintenance due to Playwright’s Model-Context-Protocol ability to adapt the test suite to changes in the application and automatically adjust test flows.
• Integration and scalability. Teams can automate test creation and maintenance, speed up test cycles, thanks to Playwright’s MCP integration capabilities with CI/CD pipelines (e.g., GitHub Actions, Jenkins) and tools like Claude Desktop or Cursor IDE.

How to Set Up MCP Server for Playwright

Setting up the Playwright MCP Server requires a few dependencies and configurations to ensure smooth operation. To get started, you need to make sure that certain prerequisites are met:

Prerequisites for MCP Server

• Node.js. As Playwright and Model-Context-Protocol Server rely on Node.js to execute automation scripts, you need to install the long-term support Node.js version 18 or later for stability and verify npm.
• A Compatible Browser Driver. You need to make sure that the appropriate browser engines are installed, because Model-Context-Protocol Server supports Chromium, Firefox, and WebKit. 
• Install the VS Code Insider build  It is important, as the Playwright MCP server requires the GitHub Copilot AI agent and certain other extensions to operate; these full functionalities are available only in the Insider build. The stable VS Code release has not yet rolled out support for them (It is my case, as a Mac user).
• Network Configuration. You need to configure firewall settings and port access to prevent connection issues, because you need to make sure your network lets many clients talk to the MCP.

Explore a couple of expert video guides to assist you through the installation process:

• Playwright MCP Server: Installing and getting started in VS Code
• Manual Testing with Playwright MCP – No Code, Just Prompts!

Installing Playwright and MCP Server

Playwright. You need to install it via npm or yarn to interact with web browsers.

init playwright@latest

Once installed, you need to verify the installation. It can be done by running:

npx playwright --version

Since Model-Context-Protocol Server is an extension of Playwright, it comes built-in with Playwright’s package. However, you need to enable the Playwright MCP Server functionality. So next, proceed with the installation of Playwright MCP Server. You can do it in a few ways:

→ Follow the Playwright GitHub Repo link and trigger the Playwright Server installation:

→ Similarly, on an official Microsoft Visual Studio Extension Page, trigger the Playwright Server:

Run the following command to install the package as a dev dependency:

npm install --save-dev @playwright/mcp@latest

Check the Playwright MCP installation in your IDE Additionally, in the settings, you can see whether the MCP Playwright functionality you need is checked or not everywhere.

Running MCP Server

Once Playwright is installed, you can start the MCP Server using the Playwright CLI. You can configure the launch file and this command separately in the package.json file and run the following command to start the server:

npx @playwright/mcp@latest

This command initializes a Playwright instance that multiple clients can connect to.

It is important to verify that the Model-Context-Protocol Server is running. After launching the server, check the logs to confirm that it’s running successfully. The logs should display connection details, including the WebSocket URL that clients will use to connect.

Connecting Clients to MCP Server

Once the MCP Server is running, multiple clients (such as test scripts, automation or monitoring services) can connect to the shared Playwright session. You can use a basic connection script for it. MCP Server might perform actions on the shared session. You can connect various clients to the MCP Server, leveraging its shared Playwright session for efficient automation or testing workflows.

Practical Understanding

• Shared Session: All clients interact with the same browser instance, so actions (e.g., navigating or clicking) affect all connected clients unless isolated contexts are created.
• Use Cases: This is useful for distributed testing (e.g., running tests across machines), real-time monitoring, or AI-driven automation (e.g., with GitHub Copilot).
• Troubleshooting: If connections fail, verify the server is running, the endpoint is correct, and there are no firewall blocks on the port.

Running tests

Once your server is configured, you can run smart test prompts. You can use your scenario in a .txt file and let MCP read the prompt file, interpret the request, generate relevant Playwright test code, and insert it directly into your project or type the prompt by yourself in the Copilot Agent window.

I asked the AI Agent to generate a Plan for the standard ToDo Playwright Demo Application  and what happened. Codepilot generated and structured 70 test cases. After that, I asked to execute this Test Plan, and the Agent provided me with a command and a proposal to run it.

Ultimately, I got this result by executing my AI-generated test plan, with the Playwright MCP server managing the entire process autonomously based on my prompts. It is a pure Vibe Testing!

Challenges and Solutions in MCP Server Playwright

Below, we are going to explore the biggest challenges you’ll face when working with the server and provide practical solutions to overcome these common issues. These are:

Issue	Possible Cause	How to Fix
MCP Server Not Starting	Playwright not installed or outdated Port already in use	You need to make sure that Playwright is properly installed and up to date. Also, you can check its version and install updates. You need to check if the default port is being used by another application and either stop the conflicting process or change the port to launch the server.
Clients Can’t Connect	MCP server not running Firewall or network blocking WebSocket	You need to verify the Model-Context-Protocol Server Status to make sure that it is running and accepting connections. If you’re running the server on a network with firewalls or restrictive security settings, you need to check the settings to make sure they don’t block the WebSocket connection between the client and the server.
Debugging is Complex	AI logic issue MCP misinterpretation App under test issue	Look at the detailed logs from both the AI client and the MCP server, which include snapshots of the page structure and network activity for each step. Apply Playwright’s built-in debugging tools, like the Trace Viewer, together with the logs created by the AI.

Playwright MCP Best Practices 

To get the most out of Playwright MCP Server, here are some best practices to take into account:

• When you’re running many clients at once, you should think about connection pooling to cut down on extra work by reusing old connections rather than to keep making new ones.
• The Model-Context-Protocol Server can handle many clients, but too many connections at once can overload it, which could cause slowdowns or failures. Knowing that, you should track resource use, like memory and CPU, to stay within your system’s limits, or set appropriate limits if necessary.
• You need to check for possible errors, like connection timeouts, pages that fail to load, or network problems, and fix them to prevent crashes or inconsistent results.
• To stop different tests or clients from clashing, you need to run each set in its own isolated space and use separate browser contexts or tabs to keep tests from interfering with one another.
• As the server can use a lot of your system’s memory and CPU, you should watch these resources while tests run to keep the server smooth. For big testing efforts, it is essential to consider upgrading your hardware or splitting the work across several machines.

Playwright MCP Integration with Test Management 

When integrating Playwright MCP with an AI-powered Test Management System (TMS) like Testomat.io, you can improve test planning, execution, and review, and make your testing efforts smarter and more automated.

• With Testomat.io, you can group and link the tests to the requirements. If tests fail, you can create an issue and fix them. Also, you are able to see the percentage of automated test coverage.
• Testomat.io allows for comprehensive and well-detailed Playwright’s test reports. Artifacts like screenshots, videos, and logs can be automatically uploaded to an S3 bucket and linked to test cases in the Testomat.io dashboard.
• Testomat.io offers direct integration with Playwright’s Trace Viewer, which can be utilized and linked in the run artifacts to examine snapshots and actions.
• When integrating Playwright MCP, you can view the history of automated Playwright’s test runs. However, it is important to mention that you need to set up the correct system configurations to make full use of this option.  

Bottom Line 

The Playwright MCP Server is a strong add-on for Playwright, which makes complex testing easier, as multiple users or scripts can work in the same session, boosting teamwork and saving resources. If you’re debugging remotely, running tests in parallel, or carrying out load testing, MCP Server helps make your automated testing process smoother. 

Contact us if you aim to add Playwright MCP Server to your testing so that your teams can manage tests well, watch progress, and create detailed reports. In addition to that, you can integrate it with a comprehensive testomat.io test case management system, which will guarantee effortless coordination among teams and make your overall testing process more efficient.

The post Playwright MCP: Modern Test Automation from Zero to Hero appeared first on testomat.io.

Based on a recent report, 76% of managers and employees noted that the lack of effective collaboration and clear communication largely contributes to workplace failure. This means that BDD is crucial for various organizations in terms of its capability to guarantee that every member of the team is on the same page and has a clear understanding of the desired software behavior. Let’s find out how the BDD framework can transform the way teams build and test today’s modern software products

What is BDD Framework?

Behavior-driven development (BDD) is a software development methodology, which has a focus on collaborative work between techies and non-techies – developers, testers, and stakeholders throughout the project’s lifecycle. With simple and natural language, teams design apps around a behavior a user expects to utilize. They write descriptions in  Given When Then  format using the user stories before any code is written to be the basis for automated test scenarios.

This BDD approach assists developers and business stakeholders in establishing a clear and common product understanding. The idea is in structuring business requirements and turning them into acceptance tests. Using tests written in plain English, all stakeholders understand and agree on the software’s expected behavior, and make sure that they develop the right software product. In BDD, teams use Gherkin language to write the script in simple words like  Given,  When , and Then . With those words, they describe the behavior of the software.

For example, Gherkin BDD framework scenario:

Feature: Product Search

  Scenario: Display search results when a user searches for a product
    Given a user is on the website
    When they perform a product search
    Then they should see search results

This test script is then turned into automated tests that check if the software behaves as expected and how it is described.

Key principles of BDD Test Framework

• Collaboration. The scenarios are written in a way that all team members – developers, testers, and key stakeholders are in the know how the system should behave regardless of their technical expertise.
• Focus on Behavior. The focus is on the users who are interacting with the product instead of how the software should be built technically.
• Common Language. Simple shared language is used across the business and technical teams so that anyone can understand business requirements and technical implementation.
• Living Documentation. BDD scenarios function as a living documentation. Since these scenarios are automated tests, they provide an up-to-date record of how the system behaves.
• Test Automation. Automating the scenarios allows teams to validate the application behavior once code changes are made. This helps catch regressions early and ensures the system behaves as expected over time.

BDD Framework Life Cycle

BDD life cycle typically includes a series of steps, which make certain that stakeholder communication and the direction of business goals or objectives are unified. Below you can find the key stages:

1. Discover. At this stage, teams collaborate with stakeholders to gain a comprehensive understanding of the project’s scope, objectives, and requirements and establish a roadmap for the project’s execution.
2. Write Scenarios in Gherkin. Teams write scenarios in Given-When-Then format to describe the product’s behavior from the users’ perspectives. These scenarios make it easier for the development teams to understand the requirements and for the QA teams to test them properly.
3. Automate Scenarios. Once scenarios are written, teams convert these plain language scenarios into automated tests using BDD test automation frameworks and tools. These tools parse the Gherkin syntax and map it to test code that interacts with the application.
4. Test. These automated tests are executed frequently to make sure that the system behavior matches the desired behavior after new code is added or existing code is modified.
5. Refactor. Teams improve existing code while maintaining behavior without changing the product’s functionality.
6. Refine and Iterate. Teams update and refine the scenarios to reflect new requirements or changes in the system’s behavior. This creates a feedback loop where the behavior is constantly validated and documented.

Why Use Playwright Java bdd automation framework?

Behavior Driven Development (BDD) with Playwright Java allows you to write tests in a more natural language, which simplifies the process of s understanding and maintaining code quality. Playwright is known as a powerful automation library which enables reliable end-to-end testing across key browser platforms (Chrome/Edge, Firefox, Safari).

Aiming to create robust, understandable, and maintainable automated tests which align with the intended functionality of your application, you can combine BDD principles with Playwright’s automation capabilities in a Java environment.

This approach will work for teams that include non-developers, such as product managers or QA engineers, who need to understand the test cases.

Playwright Cucumber Java Framework: Steps to Follow

So, let’s get started to automate! The typical technology stack for modern Java BDD projects is Java + Playwright + Cucumber together; it is a popular and well-supported choice. Here’s what each part does

• Cucumber – handles BDD-style .feature files with Gherkin syntax Given When Then
• Playwright for Java – performs the actual browser automation (clicks, navigation, input, etc.)

Java test automation framework stack includes:

• Maven – the build automation and dependency management tool for Java, is a project heartbeat.
• JUnit (usually JUnit5) – test runner; executes the Cucumber tests.

How to set up our BDD Framework?

Initially, ensure that the Java programming language development environment is installed. It is JDK 17 or higher, and Maven 3.9.3+, of course.

java -version
mvn -v

Node.js (Playwright dependency)

node -v
npm -v

If something is not installed, follow the official links to get started: Java, Playwright, Maven

So, my IDE is VS Code, and I have to install the official Microsoft Extension for Java:

By clicking the button Install you download the set of plugins, allowing you to code in Java with the Visual Studio Code editor freely now.

#1 Step: Create & configure your BDD framework project

There are two options to create a Maven project in VSCode: using the IDE UI by choosing Maven in the New Project wizard or, as in my case, through the CMD command:

mvn archetype:generate -DgroupId=com.example.demo \
-DartifactId=Demo-Java-BDD-framework \
-DarchetypeArtifactId=maven-archetype-quickstart \
-DinteractiveMode=false

Little explanation for parameters of this command:

• -DgroupId: package name base (like com.yourcompany)
• -DartifactId: Folder/project name
• -DarchetypeArtifactId: Type of project scaffold (quickstart)
• -DinteractiveMode=false: Prevents Maven from asking prompts

Once the project is created, in the editor, you will see an auto-generated basic pom.xml and project structure:

Pay attention to the Maven build notification in the bottom right-hand corner. You should agree every time after savings in the BDD framework project, anyway, to do it manually with the command:

mvn clean install

#2 Step: Configure Dependencies

The following action is adding dependencies via Maven in the pom.xml file:

• playwright
• cucumber-java
• cucumber-junit

Check the Playwright dependencies of the new version you can on the Playwright Java page

To avoid errors, you can alternatively install Playwright Java at one time:

mvn exec:java -e -Dexec.mainClass=com.microsoft.playwright.CLI -Dexec.args="install"

Similarly, you can find the required dependencies on the official Cucumber documentation page at the following link and JUnit(usually JUnit5) Dependency Page information. These configurations enable automation of step definitions and browser interactions.

Eventually, this is a minimal BDD framework example of dependencies for Playwright, Cucumber and JUnit:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.saucedemo</groupId>
    <artifactId>playwright-tests</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>23</maven.compiler.source>
        <maven.compiler.target>23</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>com.microsoft.playwright</groupId>
            <artifactId>playwright</artifactId>
            <version>1.52.0</version>
        </dependency>

          <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>7.23.0</version>
        </dependency>

       <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>7.23.0</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

  </project>

After saving pom.xml, compile the Maven build again

#3 Step: Create a Cucumber Runner

Create a TestRunner.java class file. The TestRunner.java class is like the engine that wires everything together. It tells Cucumber how to find and run .feature files.

Example TestRunner Java:

package runner;

import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@RunWith(Cucumber.class)
@CucumberOptions(
    features = "src/test/resources/features",
    glue = "steps",
    plugin = {"pretty", "html:target/cucumber-report.html"},
    monochrome = true
)
public class TestRunner {
}

The BDD framework structure of the project, as you can see in this picture, keeps logic separate and testable:

src
└── test
    ├── java
    │   ├── runners
    │   │   └── TestRunner.java
    │   └── steps
    │       └── LoginSteps.java
    └── resources
        └── features
            └── Login.feature

#4 Step: Write feature files with scenarios in Gherkin

Create  .feature files, look at the top of the code path where it is placed

Feature: Login to Sauce Demo

  Scenario: Successful login with valid credentials
    Given I open the login page
    When I enter username "standard_user" and password "secret_sauce"
    And I click the login button
    Then I should see the products page

#5:Step Map steps in Java using Cucumber step definitions

package steps;

import com.microsoft.playwright.*;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.en.*;

import static org.junit.Assert.assertTrue;

public class LoginSteps {
    Playwright playwright; // Variable playwright type of Playwright object
    Browser browser; // Represents a specific browser instance (e.g.Chromium)
    Page page; // Represents a single tab or page within the browser.

    @Before //Hook - runs before each scenario
    public void setUp() {
        playwright = Playwright.create(); //Initializes Playwright engine
        browser = playwright.chromium().launch(new BrowserType.LaunchOptions().setHeadless(false)); //Launches a visible Chromium Browser
        page = browser.newPage(); //opens a new Browser Tab
    }

    @Given("I open the login page")
    public void openLoginPage() {
        page.navigate("https://www.saucedemo.com/"); //Navigates to Log In page
    }

    @When("I enter username {string} and password {string}")
    public void enterCredentials(String username, String password) {
        page.fill("#user-name", username); //Fills in username
        page.fill("#password", password); //Fills password
    }

    @When("I click the login button")
    public void clickLogin() {
        page.click("#login-button"); //Clicks login button
    }

    @Then("I should see the products page")
    public void verifyProductsPage() {
        assertTrue(page.isVisible(".inventory_list")); //Checks of inventory list is visible
    }

    @After
    public void tearDown() {
        browser.close(); //closes browser page
        playwright.close(); // shuts down playwright engine
    }
}

#6:Step Run tests via JUnit5 runner

mvn clean test

Output

Opens browser using Playwright, navigates to login page, completes the login, verifies the dashboard is displayed and generates a basic Cucumber HTML report. Thus,

– Where can we find this BDD framework’ Cucumber HTML report?

Enter into the folder target, scroll down and launch Cucumber HTML report. It is automatically generated when Cucumber tests run with the proper configuration, namely:

@CucumberOptions(
    plugin = {"pretty", "html:target/cucumber-report.html"}
)

The Cucumber HTML Report is a simple and quite user-friendly visual representation test results of your BDD (Behavior-Driven Development) framework. It shows: Feature and Scenario breakdown, their steps in detail and result, Pass/Fail status, Error messages and stack traces, execution time. In total, it is not very informative, but it is not too bad either.

What is Playwright Test Report?

Generally, a Playwright test reports works as an extensive summary compiled after running a set of automated tests using the Playwright testing framework and indicates which scenarios passed, failed, or skipped.

With detailed reports, developers and test engineers can quickly identify the root cause of test failures and debug issues in the application code or the test automation itself. They can analyze reports to highlight areas of the application’s behavior that are not yet adequately covered by automated tests and create more scenarios.

The Testomatio Playwright Test Report Key Components

If a simple, basic Playwright or Cucumber HTML report is not enough, our solution is the perfect fit. The test management system testomat.io offers powerful Reporting and Analytics across different testing types.

In this test reporting, you can find the following information:

• Manual testing, as well as automation testing, in one place.
• Customizable test plans, selective test case execution. Easy to share it with stakeholders.
• Information on test status –  which tests have been passed, failed, or skipped.
• Descriptions of any errors, mentioning the type of error and the location.
• How long the test runs in order to identify slow tests and areas which cause performance delays.
• Information about test coverage.
• Screenshots or video recordings of the test execution to better understand the test results.
• Full test run history and clear progress tracking.
• Detailed logs that can help developers debug issues and offer visibility into browser actions, network requests, and responses.
• Moreover, actionable analytics with a wide range of metrics and insights to support decision-making.

Start by adding the Java JUnit XML plugin to the pom.xml file:

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.2.5</version> <!-- or the latest -->
      </plugin>
    </plugins>
  </build>

Sign up or log in to the TMS, then create your test project by following the system’s guidance, and then import BDD tests. Only follow the smart tips the test management UI offers.

This is the result of syncronization of manual and auto tests in the common repository.

In addition, testomat.io offers a unique feature that allows automatic converting of classical manual test cases into BDD (Behavior-Driven Development) format and importing detected steps into the Reusable Steps Database. Especially, this capability is useful for teams transitioning from traditional manual QA workflows to modern, executable BDD-style automation.

It seems this report offers a more polished presentation than the standard Cucumber report, doesn’t it?

Advantages of BDD Playwright Java framework

• Step Definition Mapping. Teams can use Given/When/Then annotations with accurate regular expressions to link Gherkin steps to Java methods.
• Playwright API Interaction. Teams can apply Playwright’s Page, Locator, and browser management APIs within step definitions in order to automate browser actions and assertions.
• Test Reporting. Teams can correctly specify the paths to Feature Files (features) and Step Definition packages (glue), which contain your automation code to set up how test results are reported and provide meaningful feedback once tests are executed.
• Parameter Passing in Steps. Teams can use capture groups in regular expressions within step annotations to pass data from Gherkin scenarios to Java methods,  because it allows them to write more reusable step definitions that can handle various data inputs from the Feature Files, and cut down code duplication.
• Assertions. With assertion methods within step definitions, teams can verify that the actual application’s behavior matches the expected outcomes defined in the Gherkin scenarios, which makes the tests reliable and verifies the software works as designed
• Selector Strategies.  With Playwright’s selector types (CSS, XPath, text-based) and reliable web elements identification, teams can automate code to accurately target and interact with specific UI elements, even after changes in the application’s structure or styling.
• Handling Asynchronous Operations.  Understanding that Playwright’s API is asynchronous and ensuring proper handling, teams can prevent their automation code from prematurely proceeding before UI elements are fully loaded or actions are completed, which contributes to more reliable and less flaky tests that accurately reflect user interactions.
• Integration with CI\CD. Teams can configure the build process to execute tests and generate reports in a continuous integration environment.

Interesting to read:

• Cucumber BDD framework for API testing with Playwright example
• Behavior-Driven Development (BDD): Meaning, Benefits & Best Practices
• Integration between BDD in Jira and Test Management System

Disadvantages of BDD test framework with Playwright Java

• Steeper Learning Curve & Complex Project Setup. If teams are new to both BDD principles and Playwright Java, it requires significant effort from them to set up the necessary dependencies.
• Complex Project Setup. Setting up the necessary dependencies (Cucumber, Playwright, Test Runner, reporting libraries) in a Java project can be more involved than setting up simpler testing frameworks.
• Too UI-centric Scenarios. Teams might fall into the trap of writing excessively detailed Gherkin scenarios that become difficult to maintain and understand. Scenarios should focus on business value, not low-level UI interactions.
• Not a Replacement for All Testing. While BDD with Playwright Java focuses on end-to-end or integration software testing from a user’s perspective, it doesn’t serve the purpose of unit tests or API tests.
• Slower Performance. Running end-to-end tests driven by Cucumber and Playwright can be slower than unit or integration tests. While Playwright is generally fast, the overhead of interpreting Gherkin and orchestrating browser actions can add to execution time, especially for large test suites.
• Maintenance Challenges. Playwright tests are susceptible to changes in the application’s user interface, meaning any minor UI modifications can break a significant number of scenarios and need frequent updates to reflect changes in UI elements, workflows, or data.
• Synchronization Issues. Web applications can be asynchronous, and handling synchronization (waiting for elements to load, animations to complete) in Playwright Step Definitions requires careful implementation to avoid flaky tests.
• Cooperation Problems. If business stakeholders are not involved in writing and reviewing feature files, the scenarios might not accurately reflect business needs.

Bottom Line: Ready To Develop The Right Product The Right Way with BDD Playwright Java?

When it comes to incorporating the Behavior-Driven Development (BDD) testing framework, organizations need to remember that it is not just a methodology; it’s a mindset. Furthermore, its adoption becomes a crucial strategy for organizations which require revolutionizing how they approach the software development process.

With BDD practice in place, you can improve communication, catch bugs early, enhance documentation, and increase test coverage. Contact our specialists if you aim to navigate software development complexities and need a working approach like BDD to develop features, which are well-understood by both technical and business stakeholders. Only by utilizing the correct BDD tools and frameworks can you get BDD’s highest potential to achieve success in your projects.

The post Playwright Java BDD Framework Tutorial appeared first on testomat.io.

In the article below, you can find information about the importance of test automation reports, reveal various types of reports, and learn how to choose the most suitable ones. Also, you can discover what tips to follow to succeed in Playwright reporting, as we consider it the most popular testing framework today.

What is Playwright?

Developed by Microsoft, Playwright is an open-source framework which is used for browser automation and testing web applications. Thanks to its ability to test Chromium, Firefox, and WebKit with a single API, teams can apply it as an all-in-one solution when conducting real-time functional, API and performance testing. Also, teams can carry out end-to-end testing by simulating user interactions such as clicking, filling out forms, and navigation.

Explore more here:

Playwright API Testing: Detailed guide with examples

Being compatible with Windows, Linux, and macOS, the Playwright tool can be integrated with major CI\CD servers such as Jenkins, CircleCI, Azure Pipeline, TravisCI, GitHub Actions, etc.

In addition to that, Playwright has broad language compatibility – supports TypeScript, JavaScript, Python, .NET, and Java – to provide QAs with more options for writing tests. In total, there is a list of key Playwright’s features:

• Cross-browser support – Chrome, Firefox, WebKit.
• Automatic waiting for elements to be ready.
• Parallel execution of tests to deliver high performance.
• Mobile device emulation and geolocation simulation.
• Easy integration with CI\CD tools.

Find more information about Playwright’s capabilities for automation testing:

Playwright Test Automation: Key Benefits and Features

What is a Test Report in Playwright?

A Playwright test report is a detailed document, which is generated after running a set of automated tests using the Playwright testing framework. It displays test results to reveal which tests were passed, failed, and skipped, and helps to uncover how well the application functions or performs.

In the Playwright test report, you can find the following components:

• Status of Tests. This component shows information about passed/failed/flaky/skipped tests.
• Error Details. This component outlines the types of errors (for example, assertion failed, timeout, network errors) and their positions within the application.
• Execution Time. Here you can discover how much time it took to run each test to uncover slow tests and performance issues.
• Screenshots. For a failed test, Playwright will automatically take a screenshot at the point of failure and provide crucial visual context.
• Videos. Playwright can record a video of the entire test execution for failed or all tests, providing dynamic information and showing what has led to a full-scale failure.
• Logs & Debug Information. Detailed logs that can help developers debug issues by providing insights into browser actions, network requests, and responses.
• Test Coverage. This component is valuable because it provides visibility into the number of tests within the coverage scope.

Indeed, Playwright reports have been designed to be interactive – with options of expanding/collapsing sections, filtering tests, and navigating through detailed failure information such as stack traces, screenshots, and videos with ease; and giving QA and dev teams an important understanding of the test’s performance in context.

Why teams need Test Automation Reports

• They see visual representations of the results of tests and can prioritize bug fixes and enhancements depending on how they affect the user experience.
• Teams are in the know about the full picture of how all the tests have been executed: they see the number of passed, failed, or skipped tests to understand how good and stable the application is.
• Thanks to reporting options, teams can get clear details of what went wrong to find and fix the main problem quickly.
• Teams can see how much of the app is being tested and which parts still need testing.
• Teams should not check the results of all tests manually to identify common problem areas in the app.
• With regular and detailed test reports, teams can monitor how well the app is doing in different tests to decide how to make their test automation better.

Different Types of Test Reporters in Playwright

When you run Playwright tests without specifying a reporter, it uses the list reporter by default. For more control, it is good practice. Specify your preferred reporter you can in the file playwright.config By default, the HTML reporter is applied.

Additionally, the easiest way to build reporters is to pass the --reporter flag with the command line. Example Playwright HTML reperter:

npx playwright test --reporter=line

Find your test result reports you can in the root folder result-reports or other if you set it.

So, let’s review the Playwright reporting types and reporter methods you can utilize to meet your testing needs.

Built-In Playwright Reporters

List Reporter

Playwright’s List Reporter provides a compact and text-based summary of the tests run. For every test that encounters a problem, it delivers the error message right near it and a call stack – this helps in figuring out what has gone wrong. While it doesn’t offer interactive features like the HTML report, its simplicity makes it an excellent choice for rapid debugging during development.

Furthermore, the List Report offers valuable information on test execution status, but without the need for a browser or complex UI. This reporter is useful for CI\CD pipelines where a simple, sequential output is preferred for logging and immediate feedback.

Line Reporter

Being a highly compact, line reporter uses a single line to display test execution results and dynamically update it as tests complete.

Line reporter is useful for large test suites, where it shows the progress but does not spam the output by listing all the tests.

It is important to mention: Line Reporter only outputs detailed information, such as error messages and stack traces, specifically when a test fails to make it very useful for developers who need quick feedback during local development or in CI environments where log details should be controlled. Overall, it prioritizes a clean console while still delivering immediate alerts for any issues.

Dot Reporter

When you run your tests in the console, Playwright’s Dot Reporter provides a highly visual representation. It uses a single dot (.) for every test that passes, so you can instantly see how things stand at any time as tests are run. If a test fails, it usually emits an ‘F’ (‘Failure’) or similar character as a warning.

Dot reporter is a good fit if you need to quickly measure overall test suite results without detailed output, making it just right to use on large projects or in the CI\CD dashboards. Its main advantage is that it offers a real-time and intuitive visual progress for your test suite.

HTML Reporter

HTML Reporter is an invaluable tool, which is used by teams to visualize test results in an intuitive and interactive web interface. After a test run, it generates a comprehensive HTML file which can be opened directly in a web browser. In our cases index.html file:

After we open the HTML file, we can see such a report in visualization Passed, Skipped or Failed tests:

The playwright HTML report gives a detailed overview of all tests, clearly displays which areas of the application were tested, and highlights the status of each test and its coverage.

For any failures, it offers detailed accounts of failures, notes error types and locations, supplemented by screenshots, videos, and powerful trace files.

What is Playwright Trace Viewer?

Playwright can record a trace of your test execution—essentially a detailed log that includes:

• Screenshots and DOM snapshots
• Network requests/responses
• Console logs
• Actions performed (clicks, inputs, navigations, etc.)

The Trace Viewer then lets you open these trace files in a visual UI for step-by-step playback. In Trace Viewer you can easily understand what exactly went wrong —maybe a timing issue, a missing element, or a slow response.

JUnit Reporter

Playwright’s JUnit Report is built to output test results in the standardized JUnit XML format that is crucial for Continuous Integration/Continuous Delivery (CI\CD) systems.

JUnit reporter produces a JUnit-style xml report.

Most likely you want to write the report to an xml file. You can see it on our screenshot in the down left corner.  When running with --reporter=junit use the environment variable:

PLAYWRIGHT_JUNIT_OUTPUT_NAME=results.xml npx playwright test --reporter=junit

In configuration file, pass options directly:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [['json', { outputFile: 'results.json' }]],
});

The generated XML file includes all the information about the test suite and cases – names, durations, and results. For failed tests, it provides essential details like error messages and stack traces, enabling automated parsing by CI tools. Its biggest advantage is that it can be used in any CI pipeline, with build servers readily able to understand the results of tests and processes controlling deployments. Although it does not provide the rich interaction of the HTML Reporter, its machine-readable format is essential to make automatic quality gates and continuous feedback work. You can download the XML JUnit Report file and upload it to various analytics tools to view data in a more refined presentation.

Multiple Reports in the Configuration File

With Playwright, you’re not restricted to a single report format, so you can meet a variety of requirements. Thanks to this adaptability in reporting, you can assign multiple reporters at once to the configuration file and define them on the console terminal. In the configuration file write:

  reporter: [
    ['html'],
    ['json', {  outputFile: 'test-results.json' }],
    ['junit', { outputFile: 'results.xml' }]
  ],

For instance, you can create a HTML report and a JSON report to receive a JSON file along with the results once you specify it in the command line or configuration file.

Custom Report

With Playwright Custom Reporter, you can tailor test result output based on the project’s unique needs. You can develop your custom reporter using JavaScript/TypeScript to transform raw test data into any format or integrate Playwright tests into existing workflows or proprietary systems that don’t support standard report formats. A custom reporter allows you to filter, aggregate, or visualize data and give a view of the results of tests, which can be reviewed by all relevant stakeholders.

For using the Custom reporter, you need to study more about the Reporter API and update the Playwright configuration file by writing its data there:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [['./my-awesome-reporter.ts', { customOption: 'some value' }]],
});

Third-Party Reporters in Playwright

Playwright allows you to integrate third-party reporters to extend its built-in reporting capabilities more extended. Thanks to these external tools Allure, Monocart, Tesults, ReportPortal, Currents, and Serenity/JS, teams can improve the reporting process by adding the following features – detailed HTML reports, real-time monitoring, and interactive dashboards; they also help teams in viewing test results and visualising them in different formats and simplify the monitoring of test performance, failures, and trends.

Max Schmitt, Open Source enthusiast, Playwright full-stack web developer, gathered all such third-party solutions for Playwright in a single GitHub repo,  Awesome Playwright.

In this repo, we are also represented

Playwright’s integration with testomat.io enables teams to see a live status before the test run has finished execution. And, a full report link will also be created to share among all parties involved as necessary.

If something fails, the execution trace, test case, and attachments can be analysed to find out what went wrong.

These reports are good for analyzing whether build compile, automated test execution, or deployment steps passed or failed.

Detect Playwright flakiness you can in a 2 way as you can see with the Analytics Dashboard widget and the AI Testing Agent. Flakiness detection helps ensure Playwright tests in the framework are dependable enough to be run automatically and frequently.

With generated reports for CI\CD pipelines, teams can create quality and deployment readiness reports automatically from their continuous integration and delivery processes. It can be achieved through the integration with the testomat.io tool.

How to Choose the Right Type of Playwright Reporter

Before selecting the type of reports, it is essential to define the needs of your team, your project scale, and the level of detail you require, and then adapt Playwright reporting to those needs. Here are a few considerations to make when deciding which type of Playwright reporting you need:

• Purpose of the Report. Your report should be driven by the main testing goals and determine the need for either quick developer feedback or comprehensive stakeholder updates.
• Size of the Test Suite. If you are testing small, a short console reporter (List or Dot Reporter) could be enough for fast feedback. But when the test suite reaches hundreds or thousands of tests, deeper reports like the HTML Reporter or specialized dashboards are invaluable to effectively explore and interpret the results.
• Environment. The testing environment heavily influences reporter choice: for local development, an interactive HTML report is ideal for immediate debugging, while for CI\CD pipelines are a good fit thirty-part Playwright reporting solution for automated parsing and quality gate integration.
• Level of Detail. The depth of understanding the results of tests is important. For detailed debugging and root cause analysis, the HTML Reporter (and it’s Traces, Screenshots, and Videos) provides every detail of any failure, up to the kind of failure, and the place in the application where it failed. If the level of detail is minimal, you can select the Line or Dot Reporter to get at-a-glance feedback.
• Data Storage Needs. If you require the historical analysis, there are reporters that generate HTML and JUnit XML files for further review. For long-term trend analysis or integration with test management systems, you can select a reporter which will sends data to an external database or service, often through a Custom Reporter. But our test management software testomat.io also support this option.
• Customization Options. If one of the regular reporters simply isn’t generating the data exactly like you need, the way the data’s aggregated, or if the data needs to be submitted to an external system, the Custom Reporter is a good option for matching the specific reporting workflows.
• Test Management Systems (TMS) Integration.  Some reports (JUnit XML, for instance) can be readily integrated with a variety of TMS to collect data in one place. So, if you need real-time monitoring of test runs, failures, and trends, you need to consider whether the report is required to directly push results to a TMS for better visibility.
• Team Cooperation. When selecting, you need to make sure the report format can be shared among team members to get a better understanding and make decisions. comprehension of the situation. If the team uses certain tools (such as Jira or Slack) to communicate with each other, then test management software might be suitable to facilitate your test result display.

Benefits of Reporting in Playwright

• Thanks to Playwright reporting, teams can get comprehensive details about test runs and quickly pinpoint the root cause of issues.
• Teams can assess the suite of results in real-time, speed up the feedback process, and maintain a continuous development flow.
• With shareable reports, teams can quickly discuss test results with business stakeholders, even those without technical backgrounds, to accelerate understanding across development, QA, and product teams.
• Teams can prevent faulty code from being deployed and ensure continuous quality thanks to automated quality gates in CI\CD.
• Teams benefit from customizable Playwright reporting options to tailor their reports to unique requirements.
• Teams can generate report files (like HTML or JUnit XML), archive results, and analyze performance, failure rates, and trends over time.

Challenges in Playwright Reporting

There are some challenges in Playwright Reporting that teams should be aware of:

• While Playwright offers custom reporters, creating interactive reports beyond the built-in options can demand significant development effort.
• Teams face difficulties in identifying key issues in the test reports in terms of including too much information in the suites of tests.
• The use of multiple environments, including various browsers and devices, can contribute to generating unpredictable results.
• Flaky tests are prone to producing false positives or negatives, which might result in inaccuracy in the reports.
• Slow page loads may cause an increase in reported execution times and impact accuracy.
• Complicated user flows and dynamic content can overwhelm reports with redundant information.

Tips for Effective Playwright Reporting

Here are some tips to follow to enhance the Playwright test reporting:

• Before executing the tests, it is essential to clearly define what you’re testing and focus on metrics which will help you determine what success looks like.
• You need to create a reporting format that is easy to interpret and helps teams resolve issues quickly. Likewise, the content can be presented in HTML format or offered as a downloadable PDF.
• For better understanding, you need to add screenshots/videos to your test reports to provide visual context and make sure they are shareable.
• You need to use CI tools to automatically trigger the report creation and distribution after each test run.

Want to Reap the Benefits from Playwright Reporting?

With good test reporting, you can turn testing data into actionable insights. Using Playwright’s reporting tools allows teams to get useful information about their results, uncover problems early, and make testing better. Thanks to diverse types of reporters in Playwright and integration capabilities, teams can integrate multiple reporters and even create custom ones to meet their different needs in testing.

If you are interested in simplifying Playwright reporting and integrating it with testomat.io for better management, do not hesitate to drop us a line and learn more about the services we provide.

The post Playwright Reporting Generation: All You Need to Know appeared first on testomat.io.

In the article, you can discover more information about the Pytest BDD framework, learn how to write BDD tests with Pytest, and reveal some considerations to help you make the most of the Pytest BDD test framework.

Why teams need the Pytest BDD framework

If your team works on Python projects, pytest-BDD will give them a sizable boost in project clarity.

• Tech teams and non-technical business executives can take part in writing test scenarios with Gherkin syntax to describe the intended behavior of software in a readable format to make sure it meets business requirements.
• Teams can verify user stories and system behavior by directly linking them to feature requirements.
• Teams can make the test automation process more scalable with pytest’s features like fixtures and plugins.
• Teams can create a solid Steps base for test cases and reuse code in other tests by turning scenarios into automated tests.
• Teams can easily update BDD scenarios as the product changes.
• Teams can get detailed test reports with relevant information about the testing efforts.

Fixtures & Tags: Why use them?

With pytest-bdd, teams can use the power of the entire Pytest ecosystem, such as fixtures and tags.

Fixtures

Marked with the @pytest.fixture decorator, fixtures are known as special functions that provide a way to set up and tear down resources required for your tests. They are very flexible and have multiple use cases – applied to individual tests, entire test classes, or even across a whole test session to optimize resource usage. There are various reasons for using Pytest fixtures:

→ Fixtures are implemented in a modular manner and are easy to use.
→ Fixtures have a scope (function, class, module, session) and lifetime that help to define how often they are created and destroyed, which is crucial for efficient and reliable testing.
→ Fixtures with function scope improve the test code’s readability and consistency to simplify the code maintenance process.
→ Pytest fixtures allows testing complex scenarios, sometimes carrying out the simple.
→ Fixtures use dependency injection (configuration settings, database connections, external services) to improve test readability and maintainability by encapsulating setup and teardown logic.

While fixtures are great for extracting data or objects that you use across multiple tests, you may not use them for tests that require slight variations in the data.

Tags

Tags are a powerful feature that helps selectively run certain tests based on their labels. They also allow teams to assign tags to scenarios in feature files and use pytest to execute tests, especially when dealing with large test suites. Tags can be used to indicate test priority, skip certain tests under specific conditions, or group tests by categories like performance, integration, or acceptance. Let’s consider the reasons for using tags:

→ You need to run suites of tests that are relevant to your current needs, like testing a particular feature.
→ You need to group tests based on their functionality, priority, or other relevant criteria to easily understand the test suite structure and find specific tests in the future.
→ You need to execute tests that match multiple tags by using logical operators (AND, OR, NOT) to precisely target the tests you want to run.
→ You need to automate the execution of specific test subsets and get customized reports based on test tags.

While tags help categorize the tests based on any criteria, their overuse can lead to a cluttered test suite and make it hard for developers to understand or maintain the code.

In fact, Pytest has limitations, but it comes with many plugins that extend its capabilities, among them the Python BDD plugin, which we are interested in at this point in the article. This plugin provides all the advantages of Python in BDD, which is why many automators love it

Getting Started with Pytest BDD

Prerequisites: Setting up the environment

If you are ready to utilize pytest-BDD, you need to make sure that all the required tools and libraries are installed. Below you can find out the steps to follow to set up the environment and start writing BDD tests:

1. 1. Install Python. You need to download the latest version from Python’s official website to get Python installed on your system. Then you need to verify the installation by running the command:

      python --version

   2. I used the optional alias python to python3 (macOS/Linux only) because I saw messages: command not found, as python3 was installed instead of python.

      alias python=python3
      

   3. I installed the package manager pip.
   4. Indeed, some test automation engineers prefer to use the Poetry library over Virtualenv. Poetry is more modern and enables management of dependencies in the global project directory without manually activating environments.
   5. Set up a Virtual Environment. At this step, you can create a virtual environment for your project to isolate it from other environments, give you full control of your project, and make it easily reproducible. Firstly, you need to install the virtualenv package if you haven’t already with pip. Once installed, you can specify the Python version and the desired name for the environment. It is a good practice to replace <version> with your Python version and <virtual-environment-name> with the environment name you want to give.

      pip install virtualenv

   6. Install pytest and pytest-BDD. At this step, you can use pip to install both the pytest framework and the Pytest-BDD plugin

      pip install pytest pytest-bdd

   7. Install Additional Dependencies. If you need additional libraries like Selenium or Playwright, you can install them as well. We need them to operate on a browser. For instance Playwright

      playwright install
      

   8. Activate virtualenv based on your OS

      source venv/bin/activate
      

   9. Create Feature Files and Steps File. The last step before writing the BDD tests is creating a structured project directory where you will keep your feature files and test scripts. Typical project structure looks like:

      pytest_bdd_selenium/
      ├── features/
      │   └── login.feature
      ├── steps/
      │   └── test_login_steps.py
      ├── tests/
      │   └── test_login.py
      ├── conftest.py
      ├── requirements.txt
      └── pytest.ini
      

How to write BDD Tests with Pytest

To write a BDD Test with Pytest, as mentioned above, you need to create a feature file and define step functions that match the scenarios in the feature file.

#1: Writing Feature File

To write feature files, you need to understand the Gherkin syntax used to describe the behavior of the application in plain English. The “given/when/then” vocabulary is pretty clear to all team members – analysts, developers, testers, and other specialists without technical background. Generally, the feature files work as living documentation of the system’s expected behavior. More information about Gherkin-based feature files can be found here.

Here is a basic example of a successful login functionality on this site https://practicetestautomation.com/practice-test-login/ 

Feature: Login functionality

  Scenario: Successful login with valid credentials
    Given the user is on the login page
    When the user enters valid username and password
    Then the user should see the secure area

#2: Creating Step Definitions

Step Definitions map the Gherkin steps in your feature files to Python functions. Pytest-bdd matches the steps in feature files with corresponding step definitions. Here is an example code for user login:

from pytest_bdd import scenarios, given, when, then

scenarios('../features/login.feature')

LOGIN_URL = "https://practicetestautomation.com/practice-test-login/"
USERNAME = "student"
PASSWORD = "Password123"

@given("the user is on the login page")
def open_login_page(browser_context):
    browser_context.goto(LOGIN_URL)

@when("the user enters valid username and password")
def login_user(browser_context):
    browser_context.fill("#username", USERNAME)
    browser_context.fill("#password", PASSWORD)
    browser_context.click("#submit")

@then("the user should see the secure area")
def check_login(browser_context):
    header = browser_context.locator("h1")
    assert "Logged In Successfully" in header.text_content()

* File test_login.py might be empty if all scenarios are loaded from a step file.

#3: Create Conftest file

Now, Playwright uses built-in fixtures like Page, and in many cases, we do not need it — Playwright provides everything.

You only need it if you want to:

• Add custom fixtures (e.g., for login tokens, DB, API)
• Change browser settings (e.g., headless, slow motion)
• Set up project-wide hooks
• Configure Playwright launch options

Our basic application is a login, so we have to create conftest.py

import pytest
from playwright.sync_api import sync_playwright

@pytest.fixture
def browser_context():
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=False)  # set True for headless
        context = browser.new_context()
        page = context.new_page()
        yield page
        browser.close()

Using a Page, you can update your steps/test_login_steps.py file

from pytest_bdd import scenarios, given, when, then
from playwright.sync_api import Page

scenarios('../features/login.feature')

LOGIN_URL = "https://practicetestautomation.com/practice-test-login/"
USERNAME = "student"
PASSWORD = "Password123"

@given("the user is on the login page")
def open_login_page(page: Page):
    page.goto(LOGIN_URL)

@when("the user enters valid username and password")
def login_user(page: Page):
    page.fill("#username", USERNAME)
    page.fill("#password", PASSWORD)
    page.click("#submit")

@then("the user should see the secure area")
def check_login(page: Page):
    assert "Logged In Successfully" in page.text_content("h1")

#3: Executing PyTest BDD

Once the feature file and step definitions have been created, you can start test execution. It can be done with the pytest command:

pytest -v

#5: Analizing Results

After PyTest BDD tests execution, you can analize, measure and review your testing efforts to identify weaknesses and formulate solutions that improve the process in the future.

If you integrate pytest BDD with a test case management system such as testomat.io, you can generate test reports, analyze them, and get the picture of how your tested software performs. 

You can debug your Playwright tests right inside the test management system for faster troubleshooting and smoother test development.

Advantages of Pytest BDD

• Pytest BDD works flawlessly with Pytest and all major Pytest plugins.
• With the fixtures feature, you can manage context between steps.
• With conftest.py, you can share step definitions and hooks.
• You can execute filtered tests alongside other Pytest tests.
• When dealing with functions that accept multiple input parameters, you can use tabular data to run the same test function with different sets of input data and make tests maintainable.

Disadvantages of Pytest BDD

• Step definition modules must have explicit declarations for feature files (via @scenario or the “scenarios” function).
• Scenario outline steps must be parsed differently
• Sharing steps between feature files can be a bit of a challenge.

Rules to follow when using Pytest BDD for Test Automation

Below you can find some important considerations when using Pytest-bdd:

• ‍You need to utilize Gherkin syntax with GWT statements.
• You need to use steps as Python functions so that pytest-bdd can match them in attribute files with their corresponding step definitions to be parameterized or defined as regular Python functions.
• You need to utilize the pytest-bdd and pytest fixture together to set up and break down the environment for testing.
• Each scenario works as an individual test case. You need to run the BDD test using the standard pytest command.
• You can use pytest-bdd hooks to generate code before or after events in the BDD test lifecycle.
• You can use tags to run specific groups of tests, prioritize them, or group them by functionality.

Bottom Line: Ready to use Pytest BDD for Python project?

With pytest-BDD, your teams get a powerful framework to implement BDD in Python projects. When writing tests in a clear and Gehrkin-readable format, teams with different backgrounds can better collaborate, understand business requirements, and build what the business really needs. Contact us if you need more information about improving your Pytest BDD workflow and its integration with the testomat.io test case management system and increasing test coverage.

The post Behavior-Driven Development: Python with Pytest BDD appeared first on testomat.io.

Before diving into the practical part, let’s explore the basics of Flask

What Flask is and how it works?

Flask is a minimalist web framework written in Python. It includes functionality, allowing developers to extend its capabilities depending on project needs. Flask does not enforce the use of a specific database or parser by default — everything can be configured, making the framework flexible and convenient for various tasks.

Why Coding in Flask?

Key advantages of using Flask in web development:

• Simplicity and minimalism. Flask is a microframework that includes only the essential components needed to build web applications.
• Flexibility and Extensibility. Unlike larger frameworks like Django, it does not impose strict structures or dependencies. It supports integration with third-party libraries and extensions, making it easy to add authentication and work with database tables.
• Built-in development server and debugging tools. It includes tools for convenient application testing and debugging. The interactive debugger helps quickly identify and fix errors.
• Active community. Flask has a large developer community, ensuring access to extensive documentation and technical support.
• Scalability. Despite its lightweight nature, Flask is well-suited for both small and large-scale projects.
• RESTful API Development. Flask is widely used for building RESTful APIs, providing a simple and efficient way to manage HTTP requests.

Testing the functionality of Flask applications is a crucial step in development. Proper testing helps identify potential errors, improve code quality, and ensure system stability.

Key differences: Unit VS E2E testing for Flask application

Testing the functionality of Flask applications is a crucial step in development. Proper testing helps identify potential errors, not only related to code quality and system stability, but it also has an impact on user experience.

In Flask applications, conducting unit tests is used for verifying individual functions and routes, while the purpose of end-to-end tests is to validate the entire application flow, ensuring that all integrated components work correctly from start to finish. E2E tests are effective in simulating real user scenarios.

AQA teams commonly use these automated testing tools for Python, such as PyTest or Playwright frameworks, allowing them to detect errors promptly before the product launch.

In this tutorial, we stay focused on testing rather than API development, we will concentrate exclusively on testing the application. The application development itself, you can learn with the video Flask Tutorial for Beginners of How to build a simple meme Python website by NetworkChuck

We are considering a simple App built with Flask that retrieves data via a public API, which is the content base of the project. Based on it, we will demonstrate how to write three types of tests, API, Unit and E2E, for a Flask application. The sample of this App you can download or clone by following the link.

Once the repository is downloaded, make sure — your Python version is 3.5 or later.

Next, you can install the necessary packages by running the following command:

pip install --upgrade pip

To run this Flask application, execute the following command:

flask run

After that, you can see a message with localhost port where your App is running

Now, when we announce the basics of Flask, it’s time to get acquainted with test framework examples and set it up for our Flask app.

First Pytest test for Flask

Pytest is a convenient Python testing framework that helps developers create higher-quality systems and confidently release them into production. With Pytest, it is easy to write both small and scalable test cases. Below you can find an example of a basic test structure using Pytest.

Pytest Project Prerequisites

First, install Pytest by executing the following command in your project’s terminal:

pip install pytest

Note: If you are working in a virtual environment, you need to activate it before installation. Using a virtual environment is the optimal solution for isolating different projects and their dependencies in Python. To know more, you can use Python Doc virtual env library

After completing the installation, connect the Pytest module. To do this, add the following line to any test file:

import pytest

Store your tests in the tests directory (folder) located in the root of the project. Pytest recommends naming test files in the format test_*.py or with the suffix **_test.py Using these standards simplifies automatic test discovery and helps avoid confusion during execution. In the screenshot below, you can see two files of tests.

In this example of the first Pytest test, we check the response from the API’s default endpoint (/). The API endpoints’ response should be 200 OK. This test fragment is used to check whether we can retrieve a meme. It ensures that the homepage loads successfully and contains the expected content.

But first, to complete the installation, connect the Pytest request module:

pip install pytest requests

Execute the test using pytest:

pytest ./tests

You can also run a specific test file by explicitly specifying its name after the Pytest command:

pytest test_app.py

After executing these commands, Pytest will automatically find all tests in the root directory or the specified file of a standard naming pattern, so you don’t need to specify file or directory names manually.

To provide further verification, we can include a content assertion validation to see if the response matches our expectations, if the client response contains the <h1>title phrase ‘Just Chill and enjoy some memes!’ and is a meme present in the <img> and tag is not empty. Therefore, we have adjusted our test accordingly. This test is a kind of Unit test — the full test_app.py file:

import requests

# Define the base URL globally to avoid repetition in each test
BASE_URL = 'http://127.0.0.1:5000/'

def test_homepage():

    # Send a GET request to the homepage
    response = requests.get(BASE_URL)

    # Assert that the request was successful (status code 200)
    assert response.status_code == 200


def test_homepage_title():

    response = requests.get(BASE_URL)

    # For example, if the homepage includes a specific title or text
    assert b'<h1>Just Chill and enjoy some memes!</h1>' in response.content
    assert b'Infinite Memes' in response.content

def test_homepage_img_meme():

    response = requests.get(BASE_URL)

    # Check if the <img> tag is present in the response content
    img_tag_start = response.text.find('<img src="')
    assert img_tag_start != -1, "No <img> tag found on the page"

    # Extract the URL from the <img> tag's src attribute
    img_tag_end = response.text.find('"', img_tag_start + len('<img src="'))
    meme_pic = response.text[img_tag_start + len('<img src="'):img_tag_end]

    # Assert that meme_pic is not empty
    assert meme_pic != '', "meme_pic should not be empty"

We have received confirmation that our test has been successfully executed! Pytest marks a successfully executed test with a green dot, and a failed test with a red letter F.

Note: If you are debugging tests through the console and want to output the response, you can use the command $ pytest -s test_*.py to write to stdout. By using the s option, you will be able to output console messages inside the tests and debug them while they are running.

When new features are added to the application, the number of tests should be increased to ensure everything works correctly. Pytest allows you to run multiple tests from one file, grouping them together. It also offers markers that allow you to assign attributes and characteristics to test functions.

Web App Testing with Playwright, Pytest Alternative

Choosing the appropriate testing framework is crucial as it ensures an efficient and seamless verification process, helping to identify and fix issues early in the development stage.

Check out the comparison table we prepared for you

Summary:

Pytest is an optimal solution for Python testing when you need a flexible and concise approach to writing tests. This tool is renowned for its versatility and supports various types of testing, including unit, functional, and API testing, as we can see just on top. It is especially helpful for writing clean, maintainable, and highly organised test code.

Playwright is a powerful tool for conducting E2E tests and browser automation, which can be very useful when testing Flask applications that have user interfaces, such as web pages; anyway, Playwright is also quite popular for API testing. Use Playwright when you need to simulate real user interactions on a web application in a browser.

Here are a few cases when Playwright should be used for testing Flask:

Testing user interaction.
Testing across different browsers.
Testing applications that heavily use JavaScript.
Simulating real-life scenarios.
Testing API with an interface.
Full application testing.
Testing interface reactions and error handling.

At the same time, Playwright is not suitable for unit testing. For checking isolated components or functions in Flask, it’s better to use clear Pytest, as well as for testing APIs without interface interaction.

For maximum effectiveness, use Playwright for browser automation and Pytest to manage the test execution, structure, and reporting.

Setting up Playwright for Flask

Prerequisites

First, let’s install Playwright. Playwright recommends using the official Playwright Pytest plugin to write E2E and API tests. This command will install Pytest-Playwright along with the Playwright command-line tool:

pip install pytest-playwright

However, it uses the synchronous version of Playwright. If you need to work with the asynchronous version, you can install Playwright without the Pytest plugin using the following command:

pip install playwright

The next important step is downloading the browsers required for Playwright automation to work. Run the command:

playwright install

Now, move on to the next stage – creating the first test.

Writing Your First Tests with Playwright

First, let’s create a file called first_playwright_test.py and add our first test scenario with the following code to it:

You can run the Playwright first test for our Flask example application by following the command:

pytest first_playwright_test.py

This test is quite simple, but it already allows us to see how Playwright works. For example, we open our URL, but the test expects <title> Infinite meme, we got the error, as in truth it is Infinite Memes. If rewrite our testing framework in Playwright it looks like below:

import re
import requests
from playwright.sync_api import Page, expect

BASE_URL = "http://127.0.0.1:5000/"

def test_homepage_title_contains_infinite_memes(page: Page):
    page.goto(BASE_URL)
    expect(page).to_have_title(re.compile("Infinite Memes", re.IGNORECASE))

def test_homepage_has_main_heading(page: Page):
    page.goto(BASE_URL)
    expect(page.get_by_role("heading", name="Just Chill and enjoy some memes!")).to_be_visible()

def test_homepage_has_image_displayed(page: Page):
    page.goto(BASE_URL)
    meme_img = page.locator("img")
    expect(meme_img).to_be_visible()
    assert meme_img.get_attribute("src") != "", "Expected meme image src not to be empty"

def test_homepage_refresh_notice(page: Page):
    page.goto(BASE_URL)
    expect(page.locator("text=auto-refresh in 15 seconds")).to_be_visible()

# ✅ API test: verify the homepage returns HTTP 200 OK
def test_api_homepage_status():
    response = requests.get(BASE_URL)
    assert response.status_code == 200, f"Expected status 200, got {response.status_code}"

Running Playwright test for Flask Application with Codegen

Playwright has an automatic code generator — Codegen. This command opens a browser window and Playwright inspector for the specified URL, allowing you to track actions.

playwright codegen http://127.0.0.1:5000/

We will use Codegen Playwright’s graphical interface to check our simple Flask web application and gradually add more tests by performing interactions on the page.

After the page loads, you can navigate across the page, and all actions with conditions will be recorded automatically. Just select the Record. The inspector window shows how the code reacts to our actions on the page. We can then experiment with the behavior we are interested in.

Codegen attempts to create reliable text-based selectors. Some results might not be optimal, for example, too generic. Thus, we made some fixes in the generated code:

In our case, we want to precisely target elements by their existing on the page, by text. This example ensures that even minor changes in the interface will be detected during testing.

Although, as you know, there are unit tests to verify this at the code level, the ability to simulate a user action — registering and uploading a file — allows detecting issues at all levels of the application.

Importance of Presenting Test Results, Reporting

Real-time reporting helps track testing efforts. It detects defects. Gives insights into which areas are okay and which need more attention. This is obvious for release planning, especially within Agile teams. When stakeholders see transparent, detailed, and accurate test results, they can improve their testing strategies and processes. In turn, it increases their confidence in the software’s quality.

Test management system testomat.io synchronises automated and manual tests for both Platwright and Pytest frameworks in one place to see the common picture in informative reports and analytics.

In addition, find Flask’s built-in capabilities useful:

Reliable error handling is an important aspect of Flask application development. In Flask, exceptions can be effectively managed using try-except blocks, preventing crashes and providing users with understandable error messages. For instance, if an API request fails or invalid data is entered, exception handling will help form a helpful response instead of causing a crash. This improves the user experience and simplifies problem diagnosis.

For further analysis and troubleshooting, logging is advisable. In Flask, logging libraries can be used to record errors, making it significantly easier to track faults by their ID. Custom error pages can also be configured to display clear messages, such as for 404 errors or internal server failures.

Another useful tool is Flask’s built-in debugger, which provides detailed error traces in development mode, helping to quickly pinpoint their causes. It is essential to remember that in a production environment, debugging mode should be turned off to prevent sensitive information leaks. Using these methods enhances the stability of Flask applications and makes them more user-friendly.

Implementing logging in Flask applications is crucial for effective debugging and performance analysis. By using Python’s built-in logging module, various log levels can be captured, from debugging and informational messages to warnings and critical errors. This allows you to record events in the application, which helps quickly identify causes of failures and track user behavior.

To set up logging in Flask, the log level and record format should be defined early in the application’s setup. Logs can be stored in files for long-term analysis or output to the console for real-time monitoring during development. It is important to include additional contextual information in logs, such as timestamps and user actions. Well-integrated logging simplifies problem detection and improves application stability, ensuring its reliable operation.

Conclusion

Mastering testing and debugging techniques is key for every developer working with Flask applications. Implementing effective testing approaches, creating an appropriate environment, and using reliable diagnostic methods can significantly improve the stability and performance of products. It is important to remember that a thoroughly tested application not only ensures a better user experience but also simplifies its ongoing maintenance and development.

Playwright is a powerful tool for reliable E2E testing, and if you are working with Python, it is definitely worth considering. By integrating Playwright with Flask testing, you ensure the seamless operation of both the backend and frontend of your application, providing users with a reliable and user-friendly experience.

The post Automation Testing Flask Application with Playwright & PyTest examples appeared first on testomat.io.

The choice between these often depends on the project’s specifics and the developer’s preferred programming language. Syntax and ease of use play an important role: some find Selenium’s approach more intuitive, while others prefer Playwright.

Each tool has its advantages and drawbacks in software development. Switching from Selenium to Cypress reveals its speed compared to the former. Test scenarios that once consumed significant time and frequently encountered errors now execute quickly and seamlessly, both locally and in CI\CD pipelines. Playwright, a relatively new tool, is an open-source framework developed by Microsoft for end-to-end testing and is currently the fastest available option.

This article provides a closer look at each of these frameworks and compares them. The information will help you choose the best tool for your project. For an introductory overview, check out the video.

What is Playwright?

Playwright supports most modern rendering browsers, such as Mozilla Firefox, Chromium, and WebKit. Its popularity is steadily increasing, enabling fast, efficient, and reliable cross-browser testing.

This tool is a Node library that automates browsers through a unified API.

The Playwright framework supports popular continuous integration servers like Jest, Mocha, and Jasmine, while also offering cross-language support, including TypeScript, JavaScript, Python, .NET, and Java.

Key Features of Playwright:

• Universal API for any web browser and platform. It operates on Windows, Linux, and macOS and supports native mobile emulation for Google Chrome on Android and Safari on iOS.
• Reduced test flakiness. Playwright automatically waits for interactive elements to become available before performing actions. It includes features like retries, trace recording, video recording, and screenshots.
• Isolated test execution. Each test runs in a separate browser context, equivalent to creating a new web browser profile without additional resource costs.
• Powerful built-in tools. Codegen allows recording test cases and saving them in any language (JavaScript, Python, .NET, Java, TypeScript). Playwright Inspector enables inspecting the page, viewing click points, and analyzing execution logs. Trace Viewer helps capture all the details necessary for analyzing test failures.

Playwright is a top-nonch solution for end-to-end testing that continues to evolve and improve.

Playwright Architecture

Playwright interacts directly through the WebSocket protocol. During test execution, the code is converted into JSON format and transmitted to the server using a WebSocket connection for command exchange. This connection remains active until one of the sides terminates it.

One of the reasons behind Playwright’s high speed is its ability to maintain an active connection for an extended period.

What is Selenium?

Selenium is an open-source framework designed for automated testing of web applications across different platforms and browsers. It supports test automation using programming languages such as Java, JavaScript, C#, Ruby, PHP, and Python.

The framework allows testers to utilize a network of browser instances for parallel execution of test scripts, significantly accelerating the testing process and the overall development cycle.

Selenium Tool Suite

1. Selenium WebDriver is an API library that is invoked from a script to execute commands in a chosen browser. Using Selenium locators, it is possible to find and define elements for creating test scripts and interact with them. For each browser, there are specific drivers that interpret the scripts written for them.

2. Selenium IDE is a user-friendly tool for recording, editing, and executing test scripts. It is available as an extension for Mozilla Firefox, Google Chrome, and Microsoft Edge. It enables quick creation and execution of automated tests without writing code by recording browser actions such as clicking elements or filling out forms and replaying them to verify the functionality of a web application. Additionally, IDE allows editing scripts, exporting them into various formats (e.g., HTML, Java, Python, or C#), and adding assertions to validate application behavior.

3. Selenium Grid is a tool for executing tests on various browsers, machines, and operating systems This functionality is especially useful for large test suites that need to be executed as quickly as possible.

In Selenium Grid, there is a central hub that manages the connected nodes. For example, one node might run Google Chrome on Windows, another Mozilla Firefox on macOS, and yet another Safari on iOS. The hub serves as the central point coordinating test execution across nodes. Nodes are the machines that execute tests. Once the nodes connect to the hub, a link is established between them. Tests are written using client libraries like Selenium WebDriver. Commands are sent to the hub, which redirects them to the appropriate nodes for execution.

By leveraging Selenium Grid, test execution can be significantly sped up by running them simultaneously on multiple machines and browsers.

Selenium Architecture

Starting with Selenium 4, the JSON Wire Protocol was replaced by the W3C WebDriver protocol, which is now the official standard for managing web browsers. Despite this, the Selenium team continues to support the legacy protocol, which can still be used via language bindings and the server.

The new protocol, known as “WebDriver W3C,” has been approved by the World Wide Web Consortium (W3C). In Selenium 4’s new architecture, the client and server communicate directly, eliminating the need for the JSON Wire Protocol. This has resulted in faster test script execution and significantly reduced instability.

Selenium Client Libraries

The W3C WebDriver protocol offers several advantages. Compared to the JSON Wire Protocol, it significantly expands capabilities. For example, the Actions API has been enhanced according to the WebDriver specification. It now allows actions such as zooming in/out and simultaneously pressing two keys with multi-touch gestures.

What is Cypress?

Cypress is an interface testing tool designed for modern web applications. It supports JavaScript and TypeScript and is used for end-to-end testing with the Mocha framework, simplifying asynchronous testing. Some time ago Cypress was a popular and promising solution considered as alternative to Selenium by many companies.

Key Advantages of Cypress:

• Ease of installation. Getting started requires only installing it via the npm or yarn packages or using the desktop downloading. There is no need to install additional libraries, dependencies, or drivers, as everything needed comes bundled with Cypress.
• Test execution speed. Cypress runs the test suite directly in the browser, and its learning curve is significantly faster than Selenium. In addition, with the built-in Chai library supporting Behavior-Driven Development (BDD) syntax, creating test scripts is quick and straightforward.
• Real-time automatic updates. After making changes to a test script, Cypress automatically reruns and updates the tests without the need for manual reloading.
• Accelerated test writing. In just a few minutes, you can begin creating test scripts, as Cypress delivers results that closely simulate real user experience.
• Automatic waiting. There’s no need to manually add waits or delays in tests. Cypress automatically waits for commands and assertions to complete before proceeding to the next step.
• Screenshots and videos. During test execution, videos of the entire test suite are recorded automatically, along with screenshots in case of failures when running headlessly. This greatly simplifies error analysis.
• User-friendly Cypress dashboard. Using the Cypress Cloud service, you can run tests in CI\CD environments and receive detailed reports on their execution.

Cypress addresses key challenges developers and testers face when working with modern applications, making the testing process fast, efficient, and convenient.

Cypress Architecture

Cypress is a test automation framework for web applications that stands out due to its unique architecture, designed to accelerate and enhance the reliability of testing. User interface testing is conducted directly in the browser, without the need for dependency on driver binary files. This eliminates network delays and speeds up the process.

Key Features of how Cypress Architecture works

Cypress operates using a Node server that constantly interacts with the browser via WebSocket, synchronizing processes and performing necessary tasks. This enables easy access to elements:

• DOM (Document Object Model)
• Local storage
• Network layer
• Window object

Such mechanism is a significant advantage of Cypress, making it a quite convenient tool for developers and testers.

Comparing Playwright vs Selenium vs Cypress Features

You can use a single tool, but it is better to utilize multiple tools simultaneously for maximum project coverage.
Sergey Almyashev
COO, ZappleTech Inc

Let’s analyze how essential functions are implemented in each of the compared frameworks.

Installation & Configuration

The setup process for test automation tools can vary significantly in terms of complexity. Selenium, Cypress, and Playwright each have unique installation requirements tailored to different testing needs. Here’s how to prepare each tool for use, so you can choose the best one based on your project requirements.

Playwright

Playwright is known for its simplicity in installation and configuration:

• Ensure Node.js is installed and its version is the latest.
• Set up a code editor, such as Visual Studio Code.
• Choose an installation method.
• VS Code extension. Create a folder, open it in the editor, find and install the Playwright extension. After that, the installation process will start automatically.
• Init command. Create a folder, open it in the editor, and run the command

  npm init playwright@latest

  During installation, select the project language (TypeScript or JavaScript).

After setup, you will have a ready-to-use .spec file.

Cypress

It is easy to set up as well, making the installation process as convenient as possible. Thanks to the structured organization of folders, you can easily create the foundation for your project.

To install Cypress, follow these steps:

1. Check if Node is installed on your computer.
2. Create a json file using the npm initcommand.
3. Install the latest version of Cypress by running

   npm install cypress --save-dev

4. Check that the json file reflects the changes.

After launching Cypress, you will immediately have access to the default .spec files for quick test initiation.

Selenium

Setting up Selenium can be a challenging task for beginners in testing! The installation process typically involves multiple steps.

1. 1. To begin, install Selenium geckodriver After downloading, extract the ZIP file to your chosen folder. Place it in /usr/bin or /usr/local/bin
   2. The next, install WebDriver. Download the latest version of Selenium WebDriver for your preferred programming language from this link For example, apply the step for Python using pip package:

      pip install -U selenium

Runner

The test runner plays a key role in running tests and exporting their results. As a library, it scans the source code directory, finds and runs test files to detect errors and failures. The runner also ensures the correct execution of tests, providing a report on their compliance with the set standards. It can run functional tests, generating results according to your configuration.

It is important to distinguish between test automation software, test run orchestrators, and execution environments.

Playwright runner

Playwright has its own test runner and also supports integration with other popular runners and works with multiple major browsers using environment variables. By default, headless testing is performed, and the results are displayed directly in the terminal. To connect Playwright to your existing test runner, you only need a lines.

Main Playwright commands:

npx playwright test

npx playwright test fileName.spec.js //Run a single test
npx playwright test tests/page/home-page/  //Run a test suite
npx playwright test home-page.spec.js --headed //Run tests in headed mode

pytest 
pytest test_login.py #Run file
pytest tests/test_todo_page.py tests/test_landing_page.py #Run specific test

Playwright supports debugging mode, which allows detailed analysis of test execution using the Playwright Inspector tool. This tool allows you to:

→ Step through Playwright API calls
→ View debug logs
→ Analyzing locators by hovering the cursor over an element during inspection

During debugging, Playwright Inspector opens a convenient interface where you can step through and examine the reasons for test failures and fix them directly during the analysis process.

Selenium runner

To execute tests using Selenium, you can use:

• Selenium WebDriver – the core component of Selenium that allows automation of actions in web browsers, such as clicks, form filling, and navigation.
• TestNG – a popular testing framework often used with Selenium. It supports test scenario creation, test suite organization, and parallel execution.
• JUnit – another widely used testing framework for Selenium with similar capabilities to TestNG, including test case creation and parallel execution.

For test execution orchestration, you can use:

• Jenkins – a continuous integration tool that automates the build, testing, and deployment processes. It integrates with Selenium, supporting parallel testing and optimizing the software delivery process.
• Maven – a powerful build automation tool that simplifies managing Selenium tests, defining dependencies, automating testing processes, and deployment.

For test execution, you can use Selenium Grid or a local Selenium server. Selenium Grid allows you to run scripts on remote machines or in cloud environments, enabling parallel test execution across different browser and operating system configurations.

Cypress

The Cypress Runner provides a convenient graphical interface for visualizing and debugging tests. It features a test list panel, an execution window displaying commands and their results, and a browser showing the real-time application being tested.

Its debugging tools include:

→ Console for logging messages and interacting with the application in real-time
→ Tab for monitoring HTTP requests and responses.

This Runner provides instant behavior analysis of the application, helping to quickly identify and resolve issues. Its interactive interface makes it easy to review test results and interact with the application.

The Cypress test status menu provides the following information:

• Test status. The status of the currently running test is displayed at the top of the panel. The test status can be one of the following: Running, Passed, Failed, Pending, or Canceled.
• Number of tests. The total number of tests in the test suite is displayed next to the test status.
• Duration. The total time it took to run the test suite is displayed below the number of tests.
• Test results. Each individual test is listed below the duration, along with its status (pass or fail) and the time it took to run.
• Command log. The command log displays a detailed list of all the Cypress commands that were executed during the test run.

The test status menu provides a quick and easy way to monitor the progress of your Cypress test suite and to identify any failing tests or errors that need to be addressed.

Orchestration of test runs:

• GitHub Actions – a tool for automating Cypress tests in a CI\CD pipeline that integrates with GitHub repositories and allows tests to run on every push or pull request.
• CircleCI – a CI\CD platform supporting parallel test execution, which speeds up the testing process by distributing tests across multiple machines.
• Jenkins – can also be used as an orchestrator for running Cypress tests.

Language/Browser Support

Record & Playback features

All three tools support record and playback functionality.

Playwright

In Playwright, test scripts are created by writing code that simulates user actions. To simplify this process, the CodeGen tool is used, which automatically generates code based on browser interactions. It records each user action, creating ready-to-use test commands, significantly speeding up test development.

Selenium

Selenium IDE allows easy recording and playback of test scripts. To use it, simply install the plugin in the browser — no additional setup is required. Unlike Selenium WebDriver and RC, Selenium IDE does not require programming code for creating test scripts. Instead, you can record browser interactions, which are automatically converted into test cases. These scripts can then be executed using the playback feature.

Cypress

Cypress Studio is a visual tool that allows you to create test cases by recording interactions with the application under test during real-time interaction with the app’s interface.

Cypress Studio automatically generates test code based on interactions with DOM elements and records them.

Element Selection

Element selection is the process of filtering all elements with a specific tag in documents and verifying whether the element you are working with is detached from the DOM.

Playwright

In Playwright, the concept of automatic waiting is implemented, which allows test execution to continue after selecting an element. Playwright automatically waits for all necessary checks to be completed. After successful completion of these checks, the requested action is performed.

Selenium

Web elements in Selenium are used to interact with DOM elements. Different types of locators are usually used to connect to page elements. Sometimes, when searching for an element, we assume it is already loaded on the page. However, this can lead to test failures if the element has not been loaded yet. To avoid this issue, we use implicit or explicit waits.

To prevent test failure, we check for the presence of an element in the DOM:

Selenium Java example:

WebElement button = new WebDriverWait(driver, 30).until(ExpectedConditions.presenceOfElementLocated(By.id("buttonId")));

button.click();

Cypress

In Cypress, the opposite occurs, as interaction with elements is done through locators. It has a built-in waiting mechanism that makes test execution more stable and reduces the likelihood of errors.

This waiting time is configurable using the defaultCommandTimeout option, which is set to 4000 ms by default. If an element is not loaded within this time, an error message appears. The value can be changed in the cypress.config.js file:

Cypress JavaScript example:

const {defineConfig} = require('cypress');

module.exports = defineConfig({

defaultCommandTimeout: 10000

});

Cypress automatically pauses for network requests to complete before moving on to the next command. In the examples below, Cypress waits 4 seconds for each command:

cy.get('.btn').click(); // Click the button

cy.focused().click(); // Click on the element with focus

cy.contains('Welcome').click(); // Click on the first element containing 'Welcome'

Parallel Execution

Parallel execution is an important feature for web browser automation, as it allows multiple test cases to run simultaneously, significantly reducing overall execution time. Selenium, Cypress, and Playwright are leading tools for parallel execution that have gained popularity among developers.

Playwright

Supports parallel test execution across multiple machines. To do this, specify the number of workers via the command line, for example:

npx playwright test --workers 4

Alternatively, modify the configuration file playwright.config.js.

Selenium

Parallel execution in Selenium means running tests from different modules simultaneously rather than sequentially. This can be achieved using TestNG. Here is an example of a configuration in TestNG XML for parallel execution:

xml

<suite name="ParallelTesting" parallel="methods" thread-count="3">

In this case, you can configure parameters such as methods, instances, tests, or classes for parallel execution. thread-count="3" ensures both test classes of our 3 tests run in parallel.

Cypress

Supports parallel execution starting from version 3.1.0. By default, it runs tests serially, but for parallel execution, you can connect Cypress Cloud to CI\CD and use the command:

C# Cypress example:

npx cypress run --record --key <your-key>

Flakiness

Flaky tests give unpredictable results, sometimes passing and sometimes failing, even when the code being tested hasn’t changed. This inconsistency creates difficulties for development teams, unlike stable tests that always produce the same results.

Playwright

The framework offers various strategies to solve test instability issues:

• Automatic waiting. Use automatic waiting to ensure actions (e.g., click or data entry) occur only after the element has fully loaded.
• Using waitFor methods. Use like waitForLoadState,waitForNavigation, and waitForSelector, to reduce instability by ensuring the page is fully loaded before interacting with elements.
• Retrying tests. Use the retry feature for cases when required elements might not be available at the initial stage of page load, causing test failures. By configuring retries in config.js the system automatically re-runs failed tests, increasing their chances of success.

Selenium

Test instability in Selenium can arise from issues with test code or test infrastructure.

To reduce instability, use the following strategies:

• Use stable selectors. For example, use classes or IDs instead of dynamic ones like XPath, as the latter are sensitive to changes in the HTML structure, which can lead to instability.
• Implement explicit waits. Replace implicit waits with explicit waits to ensure the page is fully loaded and stabilized before interacting with elements. This will help avoid test failures due to elements not being immediately available.
• Add retry mechanisms. This will retry failed checks a specified number of times before stopping. It helps reduce instability caused by temporary network or server issues.
• Ensure a stable test environment. Network and server conditions must be consistent to minimize external factors that could cause instability.
• Debug test failures. Spend time understanding the reasons for test failures. Identifying and addressing the root causes can effectively reduce test instability.
• Use reliable testing frameworks. For example, JUnit or TestNG, to organize tests and effectively manage failures. These provide built-in support for handling test failures, simplifying the debugging process.
• Rely on a reliable CI server. For example, Jenkins or Azure DevOps. They automate test execution and systematically monitor results.

Cypress

Cypress effectively handles test instability and offers various methods to address this issue:

• Assertion commands. Use checks such as .contains('element'), .should('be.visible') and .should('exist')
  before interacting with elements. This enhances the reliability of tests by verifying the presence and visibility of elements before performing actions.
• Test retries. You can configure test retries globally, for each test, or for test sets to automatically retry failed tests and increase the likelihood of their success.
• Timeouts instead of waits. Suitable for situations where elements require more time to load. By configuring timeouts at the field level, Cypress will continue the next command if the element loads before the specified timeout, improving test efficiency.
• intercept() command. Use this to wait for XHR requests to complete. By intercepting specific network requests and waiting for their execution, you can avoid test failures caused by incomplete requests.

Screenshots

Screenshots are an essential tool for diagnosing frequently failing tests, as they help identify the stages where errors occur and assist in improving tests to reduce instability.

Playwright

Offers three methods for capturing screenshots:

• for the entire page
• saving to clipboard
• capturing specific elements

This provides comprehensive visual documentation of test execution, facilitating effective debugging and enhancement of test scenarios.

Selenium

In Selenium WebDriver, capturing screenshots is indispensable for identifying errors. During test execution, the TakesScreenshot method can be used to instruct WebDriver to take a screenshot.

Cypress

Has a built-in function for taking screenshots. During test execution through cypress open or cypress run you can manually take a screenshot using cy.screenshot() To disable automatic screenshot capture on test failure, you can set the screenshotOnRunFailure parameter to false in the Cypress configuration or in Cypress.Screenshot.defaults(). This provides flexibility in managing the screenshot process according to testing needs.

Video

Video recording features are crucial for identifying errors and simplifying the debugging process, especially when tests fail. Visual recording of the sequence of test steps allows easy identification of the exact point where failures occur.

Playwright

Test video recording is managed through the video parameter in the configuration file. By default, video recording is off.

When configuring the video parameter, the following options can be selected:

• off – disables video recording.
• on – records video for each test.
• retain-on-failure – records video for each test, but deletes the video for successful tests.
• on-first-retry – records video only on the first retry of a test execution.

These options allow configuring error recording according to specific testing requirements and preferences.

Selenium

Selenium WebDriver does not have a built-in feature for recording video during test execution. However, there are extensions that allow recording all steps. These extensions help precisely identify the cause of failures at specific stages of the test. For instance, it might be the FFmpeg library. 

Cypress

Automatically records video for each .spec file. During the cypress run command, the video is recorded and saved in the folder cypress/videos path. This feature can be fully disabled by setting the video parameter to false in the configuration file.

Iframes

Iframes are HTML documents embedded within other HTML documents, typically used to add external content, such as YouTube videos or images, to web pages. If the iframe belongs to the same domain as the originating page, it can be automated using browser automation.

Playwright

To test iframes, the FrameLocator method is used, which allows access to elements within the iframe. This method handles the access logic, ensuring effective interaction.

Selenium

Provides a simple way to work with iframes. Users need to switch to the iframe window to perform the required actions and then return to the main page for further interactions. By default, Selenium works with the primary browser driver, so the driver’s focus must be changed from the main window to the specified frame element.

There are several ways to switch between frames in Selenium:

• switchTo().frame(id)– specify the ID or name of the frame as an input parameter for the switchTo()
• switchTo().frame(i)– specify the index of the frame as an argument (indexing starts from zero).
• switchTo().frame(webelement n)– pass the web element of the frame as an argument.
• switchTo().defaultContent()– returns focus from the iframe to the main page.

Cypress

Direct interaction with iframes is not supported. A plugin must be installed, and then special commands can be used to interact with elements inside the iframe. Once installed, custom commands can be used for effective interaction.

Handling API Requests

Testing includes verifying data related to elements on the API (Application Programming Interface) side – an intermediary layer between the presentation and data layers that facilitates interaction between software applications. Consequently, API testing has become an integral part of quality assurance.

Playwright

Provides access to the application’s REST API, allowing automation of requests using GET, POST, PUT, and DELETE methods.

Pay attention, we are considering Playwright API testing with a few articles on our blog already:

• Cucumber framework for API testing with Playwright example
• Playwright API Testing: Detailed guide with examples

Selenium

Primarily used for E2E UI automation and is not intended for API testing. However, through third-party tools, it is possible to partially automate API testing using Selenium Grid. This is done by opening the browser, navigating to the API endpoint, and sending HTTP requests to verify the response.

Cypress

Allows automation of API testing through thecy.request()command, which supports GET, POST, PUT, and DELETE methods. Read another article on our site — Cypress and REST API Testing

Community Support

Each framework has its own community, which helps users understand the tools and receive support in resolving issues during work.

Playwright

This is a relatively new tool on the market, so its community is still small but growing quickly. Numerous resources are available to help users get started with Playwright. The team offers detailed documentation and actively engages on GitHub, where bugs can be reported, new features requested, or contributions made to development. They respond quickly to questions and community feedback.

Selenium

Has a large and active community of users and developers. Participants have several ways to obtain support and assistance with the tool:

• Selenium documentation includes tutorials and help guides for installation or setup information.
• Forums are ideal for discussions and problem-solving. There are separate forums for different programming languages, as well as a general forum for discussions.
• Stack Overflow is a popular site for programmers, with many questions and answers related to Selenium.

This greatly simplifies the use of Selenium.

Cypress

Has a growing community that provides support through various resources and excellent official documentation. It includes an installation and usage guide, a complete API reference, and a project example library.

In addition, there are many unofficial forums and communities where user interaction occurs (more experienced users) to receive help and share experiences. Many blogs, tutorials, and video materials are available to help learn how to work with Cypress for various types of testing.

In addition to the mentioned channels, there are many user-created resources on platforms like Dev.to or Reddit, where users can get help.

Pros and Cons Selenium VS Cypress VS Playwright

Each automation testing framework has its advantages and disadvantages. Let’s examine them in more detail and in a structured way.

Playwright

It is an ideal choice for teams who value speed and efficiency in testing. Playwright is suitable for modern web applications with a need for cross-browser compatibility.

Advantages	Disadvantages
Easy setup and modern API	A relatively new tool with a smaller knowledge base and a small community
Supports multiple programming languages	Tests browser engines as opposed to full browsers
Automatic waiting, web-first assertions, and tracing feature	Lack of support for native mobile applications
High speed and efficiency	Limited browser support
Cross-browser and cross-platform support

Selenium

It is a versatile testing tool that is suitable for large projects and complex scenarios.

Advantages	Disadvantages
Support for all popular browsers and multiple programming languages	Complex setup, which can take a lot of time
Adheres to open standards and governance principles	Lack of built-in automatic waiting, which can sometimes lead to unstable tests
Various options for parallel test execution	Lower speed due to architectural features
No restrictions on the use of domains, multiple windows, or iframes	No built-in reporting feature
Large knowledge base and community support	Requires an external runner

Thanks to its flexibility and rich integration capabilities, Selenium will be useful for teams that require cross-browser and cross-platform testing.

Cypress

It is perfect for web applications that require fast and interactive testing.

Advantages	Disadvantages
Simple and intuitive API	Supports only Chromium-based browsers
Automatic element waiting and retry actions	Limited to JavaScript and TypeScript
Quick debugging capability through instant reloading	No built-in support for parallel test execution
Built-in video recording and screenshot features	For functions like XPath or reporting, third-party tool integration is required
Detailed documentation and active community support	Limited support for cross- browser testing

It will be useful for teams that use JavaScript and TypeScript and need an intuitive tool.

Playwright VS Selenium VS Cypress:

— Which Should Be Used in 2025?

To mitigate the challenges in QA automation testing services, establish a process of continuous monitoring and communication between teams and document actions in the environment. This is a challenging task, but only in cases where you do not have experienced managers in your teams.

Before spending time and effort mastering a testing automation framework, it is important to understand if this tool meets the needs of your project.

Cypress will be an ideal solution for developers working with modern web applications built on JavaScript frameworks such as Vue.js It excels in testing Single-Page Applications (SPAs) and web applications with complex interaction logic.

Selenium allows for the creation of intricate test scenarios, parallel test execution, and even working with minimized browsers. However, its speed is slower than that of Cypress, which may be a notable disadvantage for projects with tight deadlines.

In contrast, Cypress might not be the best choice for developers who require end-to-end (E2E) testing, particularly when dealing with applications containing pop-up windows or iframes. It also is not suitable for performance testing and does not support mobile device emulation, so it is better to choose another tool if these are the main testing goals.

Selenium will be useful if your project involves working with legacy browsers or systems, while Playwright is better suited for modern web applications that use the latest browser technologies. If your team requires support for multiple programming languages and integration with legacy testing platforms or complex CI\CD ecosystems, Selenium is the right choice.

Playwright will serve as the foundation for best practices if you need a quicker and more straightforward process for setting up and running tests. It is ideal for teams that prioritize built-in debugging and testing tools over-relying on third-party solutions. It is perfect for teams that need extensive browser support and advanced features. It will also be beneficial for those working with various programming languages, testing mobile applications, or aiming to ensure parallel test execution.

Although Playwright’s popularity is growing, it may not be ideal for developers who require advanced debugging tools. Furthermore, its API Request method does not support blocking tracking redirects, which makes it less practical for security researchers, ad blocker developers, and similar projects.

As for the speed of test execution across different frameworks on a static site (login verification), individual tests, and test sets, the data is shown in the diagram:

The final performance ranking is as follows:

Thus, Playwright takes the leading position in real-world scenarios. It consistently demonstrates faster execution in all three cases. In real-world situations, it showed the highest stability (least variability) in execution time.

Cypress has a longer startup time compared to other tools, which affects the speed of executing short tests, although this effect is smaller for longer tests. In longer test sets, Cypress’s speed is comparable to Selenium’s, which is the benchmark for E2E testing.

For now, choose the framework that best aligns with the specific requirements of your project.

Trends in Playwright Usage

Playwright is continuously improving, and in 2025, the following trends are anticipated:

• Integration with Artificial Intelligence. This will facilitate automatic test creation, result analysis, and the prediction of potential issues, improving testing accuracy and efficiency.
• Broader support for various platforms. Playwright is expected to expand its compatibility with different operating systems and browsers, including mobile platforms, allowing web application testing across an even larger variety of devices and browsers.
• Improved speed and performance. Playwright developers are working on optimizing test speed and reducing setup time, which will accelerate test execution.
• Easier integration with CI\CD. This will automate the testing process and significantly speed up defect detection.
• Enhanced documentation and community support. This should simplify the tool’s learning curve for new users.

Today, Playwright will keep incorporating cutting-edge technologies and improvements to maintain its dominant position among automated web application testing tools.

Summary

In modern programming, test automation is often achieved using tools such as Selenium, Cypress, and Playwright. Each of these tools has unique advantages and limitations, making them suitable for various testing needs.

Selenium, being one of the oldest and most widely recognized frameworks, provides excellent support for multiple programming languages and browsers, making it essential for end-to-end testing.

Cypress offers a contemporary approach to automation, emphasizing direct browser testing, user-friendliness, and seamless integration for unit testing.

Playwright, on the other hand, demonstrates flexibility with support for multiple browsers, parallel test execution, and the automation of complex tasks like mobile device emulation.

The popularity of Playwright is growing. However, Selenium remains the primary choice for many teams due to its time-tested stability and wide functionality, especially with Java or Python languages.

The choice of test automation tool ultimately depends on the specific needs of the project, the developers’ experience, and the features of the application that needs to be tested.

The post Playwright VS Selenium VS Cypress: A Detailed Comparison appeared first on testomat.io.

However, in addition to their common characteristics, they have significant differences. For example, Puppeteer is focused on automating Google Chrome and Chromium browsers, while Playwright offers cross-browser capabilities.

Next, we will take a closer look at the features of each framework and determine which one is best suited for the testing process in your project.

What to Consider When Choosing a Library

If you are at the stage of selecting a tool for test automation, rely on the following criteria:

1. Support for Required Test Types. Each tool, due to its functionality, may not be suitable for all tasks. Therefore, define in advance what type of tests you will need.

2. Learning Curve and Ease of Use. Before starting, evaluate the community of the specific tool, its documentation, and guides. As for ease of use, test the user interface, API, etc.

3. Compatibility and Integration. Identify which browsers and platforms the tool supports. It is also essential to evaluate how seamlessly it integrates with third-party services, such as CI\CD pipelines or testing frameworks.

4. Functional Features and Performance. Assess whether the library offers functionalities that can enhance testing efficiency. This could include parallel test execution, working with headless browsers, etc.

5. Scalability. If the number of test cases on the project is expected to increase over time, ensure that the library can handle the growing load.

6. Cost and Licensing. There are open-source libraries and paid solutions on the market. Evaluate the available functionality and decide whether a free tool is sufficient for you.

We hope these recommendations will help you choose the right tool for web automation tasks. Now it’s time to discuss the capabilities of the two specific frameworks – Puppeteer and Playwright – and finally make your choice.

What is Puppeteer? Key Features of the Framework

Puppeteer is a Node.js library created by Google in 2017 and is currently maintained by the Chrome dev team. It provides a set of APIs for interacting with web pages. Among the key browser automation tasks are web scraping and automated testing in both headful and headless modes.

On the official website, the tool is defined as:

Puppeteer is a JavaScript library which provides a high-level API to control Chrome or Firefox over the DevTools Protocol or WebDriver BiDi.

Puppeteer framework supports easy integration with popular JS testing environments. Together with its extensive functionality, this makes it a flexible tool suitable for different types of testing.

Here are the prominent features of Puppeteer

• Simulating User Actions. The tool can replicate a wide range of user interactions, such as clicking buttons, filling out forms, handling alerts, navigating pages, and more. This makes it ideal for end-to-end testing, as it ensures the validation of all user flows within a product.

• Full Automation. Puppeteer automates the entire workflow, from opening web browsers to interacting with elements on web pages.

• Testing in Headless Mode. The tool supports working with headless browsers, meaning browsers without a visible user interface. This improves testing speed. However, for certain tasks, such as debugging or visual inspection, tests can be run in headful mode.

• Simple Interaction Methods with DOM Elements. Puppeteer provides simple methods for interacting with DOM elements, including clicking buttons, entering text, selecting options, and more, making it an excellent choice for UI testing.

• Wait for Elements. The tool can halt execution until specific elements appear in the DOM, which is particularly helpful for applications with asynchronous processes where elements are dynamically loaded.

• Navigation Control. Puppeteer enables programmatic navigation between web pages, ensuring that the application performs correctly during different user journeys.

• Network Interception. The tool can intercept and modify network requests and responses. This feature is valuable for testing how an application reacts to different network conditions, such as slow connections or server failures.

• Simulating API Responses. Puppeteer allows you to simulate internal scenarios without needing a real server. It generates mock API responses, enabling QA teams to conduct quick, isolated tests.

• Screenshots. The framework allows users to take screenshots of entire pages or specific elements. The former can be compared to baseline images to detect visual changes, while the latter can be used to check individual components. This enables visual regression testing.

• Performance Measurement. Puppeteer is excellent for performance testing, as it can measure various metrics like page load times or time to interactivity. It also collects browser performance metrics, such as CPU usage and memory consumption.

• Cross-Browser Testing. This is not Puppeteer’s strength, as it is primarily focused on Chromium and has limited Firefox support. Therefore, if you need to test applications in other modern browsers, this tool may not be suitable. Other frameworks would be a better option.

• Device Emulation. The tool allows you to emulate mobile devices and tablets, including their screen sizes and geolocation data. This enables testing of specific features and designs. Additionally, users can modify the User-Agent string to simulate different browsers and devices.

• Parallel Testing. Puppeteer provides different browser contexts, making parallel test execution possible. This option, combined with headless mode, speeds up the process and makes Puppeteer suitable for CI pipeline integration.
• Debugging Features. Puppeteer users can view browser execution in real-time, provided they use headful mode. It allows pausing test execution to debug script issues. If you choose this tool for your project, you will have full control over browser functions through access to the Chrome DevTools Protocol.
• Automatic Error Handling. Puppeteer allows handling common errors (such as page timeouts, elements not found, etc.) without manual intervention. Tests can be retried or marked as failures.

All these features make the library the number one choice for many teams. Let’s explore its advantages and disadvantages so you can make an informed decision.

Puppeteer: Key Advantages and Disadvantages

The Advantages of Puppeteer include high testing speed and extensive capabilities for QA teams. Let’s take a closer look at them

1. Cross-platform support. The tool works on various operating systems – Windows, macOS, and Linux. This makes it compatible with many environments.

2. Ability to conduct various types of testing. Puppeteer allows for E2E, API, UI, and performance testing. We will discuss this in more detail later.

3. High testing speed. This is facilitated by some of Puppeteer’s features, including testing in headless browsers, automatic error handling, parallel testing, API response simulation, and more.

4. Integration with popular JavaScript testing frameworks. Puppeteer integrates easily with Jest, Mocha, Jasmine, and other frameworks. This allows for easy writing and effective management of test cases.

5. User-friendly interface. The tool offers a convenient API. It allows automating browser actions with minimal code.

6. Extensive community support. Puppeteer has a large user base, an active GitHub repository, and detailed documentation. Any questions can also be asked on numerous forums, chat groups, and discussion platforms.

7. Moderate learning curve. Since Puppeteer is based on the DevTools protocol, it will be easy for anyone familiar with Chrome Developer Tools to get started with it.

In addition to the above advantages, the library has some Puppeteer Drawbacks:

1. Limited browser support. Puppeteer is not exactly a cross-browser tool. It is focused on automating Chromium and Google Chrome. Although the latest version does offer experimental Firefox support.

2. Dependency on Node.js. Since Puppeteer is a Node.js library, a corresponding environment is required for it to function. Therefore, this tool is not suitable for teams that do not use Node.js as their main technology stack.

3. Limited mobile device emulation capabilities. The tool offers this feature, but it is quite limited. For example, other frameworks (e.g., Playwright) can emulate more devices and handle touch gestures more comprehensively.

4. Challenges with testing interactive applications. Puppeteer can simulate most user actions. However, when it comes to testing complex scenarios, additional configuration may be required.

5. Lack of built-in support for multiple browsers. Running tests in different browsers, such as Chrome and Firefox, requires configuring multiple instances of Puppeteer.

Consider all the disadvantages of Puppeteer and evaluate how critical they are for the quality of work on your project.

Using Puppeteer Legally

Puppeteer is often primarily associated with complex web scraping activities. The legality of such activity can be questioned depending on various factors:

• the terms of service of the target website
• the goals of scraping (copying content may violate copyright laws)
• the specific laws of a country (for example, in the US, scraping can lead to civil or criminal sanctions under the Computer Fraud and Abuse Act)

For the use of Puppeteer to be legal in this context, it is necessary to limit the collection of publicly available data or obtain permission from the website owner.

However, it is also important to remember that another legal use of Puppeteer is web testing. Let’s explore in detail what QA activities the tool supports

Testing with Puppeteer

Puppeteer allows teams to apply a flexible approach to testing various aspects of application performance. Look at how the tool can help ensure premium quality for the final product.

Testing Mobile Progressive Web Applications (PWA)

In the context of testing PWAs, Puppeteer focuses on ensuring that these applications behave like native mobile apps. With the tool, you can test the following features of progressive web applications:

• the operation of service workers under different conditions
• offline functionality (without an Internet connection)
• the correct installation of the app (e.g., displaying the “Add to Home Screen” prompt)
• the functioning of push notifications
• correct use of the PWA manifest file

End-to-End Testing

Puppeteer allows you to simulate real user interactions with the application. The simulations can include any actions, from opening web browsers to filling out/sending forms and navigating web pages.

E2E testing with Puppeteer ensures that all processes in the application work flawlessly, as expected.

API Testing

The tool is primarily used for browser interactions. However, another key feature of Puppeteer is simulating network requests and API calls. This should be done using the page.setRequestInterception(true) method. It allows for checking the correctness of requests and the adequacy of responses.

Performance Testing

Puppeteer is well-suited for performance testing. It integrates easily with powerful tools such as Lighthouse to conduct performance audits. You can also use Puppeteer’s own APIs to gather relevant metrics:

• time to load web pages
• time to first meaningful paint (FMP)
• time to interactive (TTI)
• response speed across different devices.

UI Testing

The Puppeteer library can also be used for testing UI components. With the automation of user actions, you can check the responsiveness, visibility, and interaction of elements under different conditions.

The advantage is that the tool offers advanced features that allow you to optimize the execution of tests of this type. This includes taking screenshots and recording videos during the testing process.

Next, let’s explore testing with Puppeteer through a practical example.

Your First Test With Puppeteer: Code Example

Working with Puppeteer starts with installing it. You’ll need the npm package manager for this:

npm install puppeteer

Then, set up the environment by following the guide on GitHub:

node server.js

Next, here’s a code example to test the functionality of logging into a website. This requires performing two steps:

1. Create the file test-login.js
2. Add the following code:

const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();
  const timeout = 5000;
  page.setDefaultTimeout(timeout);

  try {
    await page.goto("http://localhost:8000");
    await page.waitForSelector(`#fname`, {timeout, visible: true});

    await page.type(`#email`, `your email`);
    await page.type(`#password`, `your password`);

    await page.screenshot({path: "./login.png", fullPage: true});

    await Promise.all([
      page.click(`input[name="loginSubmit"]`),
      page.waitForNavigation()
    ]);

    const url = await page.url();
    if (url !== 'http://localhost:8000/login') {
      throw new Error(`The URL was not http://localhost:8000/login`);
    }

    console.log(url);
  } catch (err) {
    console.log(err);
  } finally {
    await browser.close();
  }
})();

Code example for testing login functionality

What functions did we use?

• type()– for entering text into the corresponding field;
• click()– for clicking the “Login” button;
• waitForNavigation()– for waiting for the navigation to a new browser page.

Now, let’s look at how to automate testing with Puppeteer using the above example. To do this:

1. Create the file test-login-automated.js
2. Add the following code:

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch({headless: false});
    const page = await browser.newPage();
    const timeout = 5000;
    page.setDefaultTimeout(timeout);

    try {
        await page.goto("http://localhost:8000");
        await page.waitForSelector(`#fname`, {timeout, visible: true});

        // Fill in email and password fields
        await page.type(`#email`, "your email");
        await page.type(`#password`, "your password");

        await page.screenshot({path: "./login.png", fullPage: true});

        await Promise.all([
            // Click on login button
            page.click(`input[name="loginSubmit"]`),
            
            // Wait for page to navigate to the next page
            page.waitForNavigation()
        ]);

        // Get url of the page
        const url = await page.url();
        if (url !== 'http://localhost:8000/login') {
            throw new Error(`The URL was not http://localhost:8000/login`);
        }

        console.log(url);
    } catch (err) {
        console.log(err);
    } finally {
        // Close browser window
        await browser.close();
    }

Code example for automating tests with Puppeteer

3. Run the code using the command:

   node test/login.js

Congratulations! You have just written your first test with Puppeteer. Now, let’s explore how you can optimize the testing process with this library.

How to Optimize Your Tests With Puppeteer?

We’ve gathered the best practical tips to improve the speed and efficiency of testing with Puppeteer:

Do not overuse the waitFor* Excessive usage can slow down the testing speed.
Limit the number of actions performed in each test. This will help improve performance.
Run tests in parallel and use headless mode. This will also positively affect performance.
Pay attention to settings and updates. Properly configured test runners and code updated to the latest Puppeteer version ensure tests run correctly.
Study the documentation carefully. In addition to the library usage guide, it’s useful to review the Headless Chrome
Avoid using the library for web scraping, as it may be illegal.

Follow these tips, and testing with Puppeteer will meet your team’s expectations.

When Is It Worth Choosing Puppeteer?

Naturally, when selecting a tech stack for specific tasks, you should focus on the specific project requirements. However, in this section, we have gathered some general signs that Puppeteer is the perfect choice for your team.

• Simple projects focused on automating Chrome/Chromium. If you are working with these browsers, Puppeteer will be the best option for you.
• Projects focusing on Headless Mode. The tool supports working with headless browsers. This makes it an ideal choice for teams that value speed and high performance.
• Teams interested in a quick start. This library has been around for a long time, so it has community support, detailed documentation, and guides. Therefore, a steep learning curve is not an issue with Puppeteer.
• Sufficient resources available for the project. Testing with this tool is quite resource-intensive for your machine. Keep this in mind when choosing a library.

If your project does not involve working with dynamic content, you can consider using simpler frameworks (like Jest for unit testing). They require fewer resources and therefore can be more efficient.

What Does the Framework Offer to Automation Engineers?

Playwright is also a modern Node.js library that has been meeting the needs of users for testing and browser automation since 2020.

According to information provided on the official website of the framework — Playwright Test Automation framework was created specifically to accommodate the needs of end-to-end testing. However, this does not mean it only supports this type of testing.

This library stands out from Puppeteer due to its variety of supported browsers and a stable set of features, making it an ideal framework for complex projects.

Here are the key features of Playwright:

• Cross-browser testing. Playwright can work with Chromium, Firefox, and WebKit (Safari). This ensures that your application works consistently across different web browsers.

• Cross-platform. The framework supports various operating systems, including Windows, Linux, and macOS. Testing with this tool ensures that web applications will function properly on different devices.

• Automatic waiting. Before performing any action, Playwright waits for the elements to become active. This protects users from unstable tests.

• Web-first assertions. The framework allows you to repeat checks until all necessary conditions are met.

• Browser contexts. Playwright can work with multiple browser contexts simultaneously. This allows you to create isolated tests and speed up the testing process.

• Codegen. This feature enables you to generate tests by recording user actions without writing code. The tests can be recorded in any of the supported programming languages.

• Real device. Playwright provides built-in emulation of mobile devices and the ability to test responsive web designs.

• Network interception. You can intercept and manipulate network requests. This feature is particularly useful for testing edge cases.

• Support for headless and full mode. This makes testing with Playwright highly flexible, essential for running tests in different environments.

• Playwright inspector. This feature is key to obtaining detailed information about executed tests. It allows you to inspect pages, generate selectors, view step-by-step test execution, check execution logs, and more.

• Trace Viewer. This feature is essential for identifying the cause of failures during testing. Users have access to screen recordings, DOM snapshots, test code, and more.

• Imitating real user actions. The tool can simulate real user interactions with web pages, such as filling out and submitting forms, dragging elements, and more.

• Visualizing test results. Playwright supports creating screenshots of the entire page or individual elements, as well as recording videos. These tools help testers quickly understand the issue and pass the bug to developers for fixing.

To learn more about Playwright’s capabilities, check out this video: Test Automation with Playwright

Such multi-functionality and versatility often serve as arguments for choosing this framework for test automation. Let’s look at other advantages of Playwright and potential challenges when using the tool.

Pros and Cons of Playwright

Users often choose Playwright for testing on their projects because of these advantages:

1. Support for various testing environments. By working with popular web browsers and operating systems, Playwright ensures that the application works uniformly across different devices.

2. Working with headless and headful browsers. This allows the framework to be used for various tasks. Headless mode is ideal for speeding up testing and integrating into CI pipelines, while headful mode is suited for debugging and manual interaction.

3. High testing performance. This is facilitated by the ability to run tests in parallel and the support for browser contexts.

4. Reliability of tests. This is achieved by the automatic waiting feature, which eliminates the need to create artificial timeouts, a common cause of unstable tests.

5. Integration with popular JavaScript testing frameworks. Playwright can easily integrate with Jest, Mocha, Cypress, and other frameworks, simplifying work with test cases.

If you are planning to choose this framework for testing on your project, it is also important to familiarize yourself with the disadvantages of Playwright:

1. Limited community support. Since it is a relatively new tool, its user base is still developing. This also affects the availability of guides and documentation – they might be difficult to find online.

2. Steep learning curve. Beginners in the field of browser automation may require extensive training, especially when it comes to complex web automation tasks like network interception or working with multiple browser contexts.

3. Limited automation capabilities. Playwright is focused on automating web applications. If you are also working with mobile or desktop solutions, you may want to choose another tool.

4. Dependency on Node.js. Playwright requires Node.js to run. Therefore, it is mainly suitable for users working with JavaScript/TypeScript.

Playwright is a powerful library designed for automating modern browsers. It has its advantages and disadvantages, which should be considered when choosing the tech stack for your project.

More How to setup first Playwright project & details of framework follow our articles:

• Playwright AI Revolution in Test Automation
• Cucumber framework for API testing with Playwright example
• Playwright Test Automation: Key Benefits and Features

For Which Projects Is Playwright the Best Choice?

When should you choose Playwright to optimize testing? The following scenarios include:

• The project involves cross-browser testing on Chromium, Firefox, and WebKit.
• You need device emulation.
• The project involves complex interactions, such as network interception, multi-page scenarios, etc.
• You are interested in the highest possible testing speed.
• The team uses JavaScript/TypeScript and would benefit from Node.js support.

— Did you recognize your team in the criteria above? Feel free to choose Playwright – it will help achieve your project goals.

Puppeteer vs Playwright: Comparative Analysis (Table)

To systematize the information presented in this material, we offer you a comparative table of Playwright vs Puppeteer.

Supported OS	Windows, macOS, Linux	Windows, macOS, Linux
Browser Support	Chrome/Chromium limited Firefox support	Chromium, Firefox, WebKit (Safari)
Cross-Browser Testing	Limited support – within Chrome/Chromium and Firefox	Full support for all browsers mentioned above
Mobile Device Emulation	Supported	Supported, more user-friendly and reliable feature
Parallel Test Execution	Supported	Supported
Language Support	JavaScript/TypeScript, Python, C#, Go	JavaScript/TypeScript, Python, C#, Java
Headless Mode	Supported	Supported
Network Interception	Basic support	Advanced support with network request mocking and intercepting responses
Optimal Project Type	Simple Chrome/Chromium-focused projects	Cross-browser testing, advanced test automation
Primary Use Case	Chrome automation, simple web automation tasks	Complex workflows (parallel testing, user emulation, etc.)

Why Use Puppeteer and Playwright in Agile?

Using Puppeteer and Playwright in Agile projects can bring teams several benefits. These primarily include high testing speed, process automation, and optimal performance.

Here’s how these libraries can help achieve Agile development goals:

• High test execution speed. Agile methodology relies on iterative development and requires quick feedback. The tools in question allow for high testing speed, helping teams monitor application functionality during each iteration.

• Continuous Integration (CI) support. Puppeteer and Playwright integrate seamlessly into CI/CD pipelines. This enables automated testing during each build or deployment, ensuring continuous testing throughout the project lifecycle.

• Ensuring high application quality. Both libraries support testing across various OS and provide device emulation capabilities. Playwright also enables cross-browser testing. This ensures that QA teams can verify the product’s performance on any device.

• Support for the Shift-Left practice. Both tools allow teams to shift testing to earlier stages of the SDLC, aligning with Agile methodology principles.

• Process automation. Puppeteer and Playwright support test automation, helping reduce human errors and minimize the risk of regressions.

The Node.js libraries discussed in this article are well-suited for Agile projects. They provide teams with rapid feedback, facilitate frequent releases, and help create high-quality digital products.

Final Thoughts

— Which framework for web automation tasks is the best fit for your team? To answer this question, you need to consider the project requirements, tool functionality, ease of use, cost, and other criteria…

In this material, we have reviewed two Node.js libraries frequently used in Agile projects – Puppeteer and Playwright. Both are powerful tools with extensive capabilities in the field of web automation.

The post Puppeteer vs Playwright: Choosing the Best Library for Browser Automation appeared first on testomat.io.

Achieving these goals is impossible without a well-structured testing strategy, an important part of which is test automation. Moreover, the reality of today is such that automating tests is more optimal with the power of AI.

Artificial intelligence and generative AI may be the most important technology of any lifetime,

says SalesForce CEO Marc Benioff

— How are teams using AI testing technology in their workflow when using Playwright

Let’s explore further!

Codegen Playwright

Codegen — is a powerful built-in Playwright tool that automatically creates test cases by recording user interactions with a web application or website. Its usage like an AI tool is the most predictable. Playwright Codegen is not only fast, but also quite a smart testing tool

To get started with Playwright Codegen, you should run the next command in the terminal:

npx playwright codegen

This will open two windows: a browser, where you will perform various actions, and the Playwright Inspector. The latter will display the Codegen window filled with the code generated as a result of your interactions. It will look like this:

All these actions take just a few seconds. This speeds up the execution of testing tasks and allows automation engineers to implement best testing practices in their projects.

Anyway, it is important to remember — Playwright Codegen is not perfect, and all the code it generates should be double-checked for alignment with the team’s needs.

AI testing & Automation with ChatGPT

An alternative to classic test automation with Playwright Codegen is generating test code with AI. To do this, in a dialogue window – for example, the well-known is a GPT chat – plain text instructions should be entered with its interface. They should contain information about what exactly needs to be tested, which device configuration to use, how to write a particular piece of code or improove it etc.

Of course, the generated by AI Playwright tests may contain some inaccuracies in your testing scripts. For example, your AI code may use locators that are not recommended by the framework. These include page.click() and page.fill() Using the same plain text commands, you can point out the inaccuracies to GPT and ask it to correct them to the proper version – in our case, role-based locators.

To avoid constant revisions, it is advisable to include preliminary plain text prompts in the request to generate the Playwright AI tests. A small recommendation – example prompts can be found on cursor.directory:

Thus, to successfully generate Playwright tests using AI, you should:

use a specialized prompt to help the tool understand what it is dealing with;
verify the accuracy and relevance of the generated code;
provide as much context as possible (e.g., JavaScript and HTML files – anything that supports your app).

Given that modern web applications have highly complex architectures, fulfilling all these requirements can be challenging. Therefore, Playwright Codegen remains a more convenient option for the rapid creation of tests.

GitHub Copilot for Playwright Code Generation

Copilot AI is a powerful tool that analyzes the context of your code and then suggests complete Playwright test scripts that you can use for test creation, debugging, refactoring and maintenance of your code. It is seamless with all Microsoft Products.

For instance, Copilot identifies redundant lines, incorrect selectors, unused variables, or better Playwright-specific API methods to enhance maintainability and performance. Copilot AI may be trained on public code or your own. Moreover, Copilot’s alternative AI code can simplify intricate test scenarios, fix asynchronous issues or propose ways to handle flaky tests, saving time spent thinking up the best solutions at times.

Testomatio AI test management for your workflow

Recently, testomat.io team developed a couple of new AI features from testomat.io, including AI-powered test case creation with advanced analytics for QA test managers. These improvements provide smarter test management, increased productivity, and full project control.

It suggests optimized tests with AI for popular test automation frameworks like Cypress, Jest, Cucumber, PyTest, and JUnit etc, including the Playwright framework. AI feature automatically generates test case ideas based on the content of the test suite. Users can review the suggested tests, select the relevant ones and add them directly to the suite or reject them and re-generate the new ones. This accelerates test case creation, enhances test coverage by identifying overlooked scenarios, and reduces manual effort while maintaining high testing quality.

Why Testomatio AI Test Management insights worth implementing?

• Increased productivity. Save time with faster synchronization of your manual and automated test sets.
  Improved test quality. AI Flaky test detection identifies patterns in test execution to pinpoint tests that pass or fail inconsistently.
• Intelligent test maintenance. Maintaining test cases in dynamic environments can be tedious. Testomat.io’s AI identifies duplicates, obsolete or redundant test cases or test steps based on them. Test history trends over time and Analytics may empower teams about gaps in test coverage after updates.
• Faster Releases. Predictable analytics and smart prioritization help identify and address high-risk areas in test strategy promptly.
• Seamless Collaboration. Enable non-technical stakeholders to contribute using plain language test creation.

We provide a Custom AI Provider integration. It enables companies to use their preferred AI provider, such as OpenAI or Anthropic, for all AI-driven features. This feature offers flexibility in leveraging AI capabilities while ensuring compliance with data-sharing policies and regulations.

Also, as an alternative, you can use third-party AI-based tools designed to optimize work with the popular framework.

Other Third-Party AI Tools work with Playwright

Playwright itself does not provide users with artificial intelligence-based functionality. As a result, its enthusiasts have to turn to third-party tools to optimize web automation with their favourite library.

QA teams who find Codegen’s capabilities insufficient may consider AI-based automation testing tools like Auto Playwright or ZeroStep. Below you can see an example of a piece of code generated by ZeroStep AI-based tool.

Now, take a look at their features and advantages

ZeroStep

ZeroStep – is one of the most effective testing solutions that leverages the power of GPT-3.5 and GPT-4 for the rapid automation of Playwright tests.

Using the platform involves writing instructions in natural language. Simply tell the tool how you need to interact with the website, and AI will handle the rest for you!

To get started with ZeroStep, use the command:

npm i @zerostep/playwright -D

ZeroStep Playwright AI features for optimizing web testing

• Test development without writing code. All the user needs to do is interact with the application via the UI. The tool will generate the tests automatically.
• Cross-browser and cross-platform testing. The platform supports testing in all popular browsers (e.g., Chrome, Firefox, Safari) and on various platforms (Windows, macOS, Android). This guarantees that the digital solution performs consistently across various environments.
• Self-healing tests. The tool automatically adapts tests to minor changes in the product, minimizing the need for manual intervention.
• Seamless integration with third-party platforms. ZeroStep easily integrates with CI\CD solutions such as GitHub Actions, GitLab, Jenkins, etc.
• Detailed reporting and analytics. The platform provides users with comprehensive reports featuring visualized results. Additionally, information on test coverage, performance, and other analytics is available.

Advantages of ZeroStep AI

• Affordable pricing. Users get access to 500 free AI() function calls per month. If you need to increase the limit, the cost starts at $20.
• Optimized collaboration. The no-code approach allows non-technical team members to participate in testing.
• Increased accuracy in software testing. Smart test maintenance adapts tests to changes in the application. This helps keep them up-to-date and prevents inconsistent test results.
• Fast Feedback. Real-time test execution and integration with CI\CD pipelines enable teams to respond to issues promptly.

Auto Playwright

This is another tool that allows teams to integrate artificial intelligence into their Playwright workflows. It serves as an intelligent intermediary between the user’s plain text instructions and the test code.

This approach ensures high-quality software products in today’s dynamic digital landscape.

Setting up and configuring Auto Playwright is straightforward. To get started, follow these steps:

1. Install the dependency using the command:

   npm install auto-playwright -D

2. Create an OpenAI API key (you will need to register on the OpenAI platform for this).
3. Export the OpenAI API token by running the command:

   export OPENAI_API_KEY='sk-...'

4. Configure the environment variables following the guidelines provided in the official documentation for the tool.

After completing these steps, you can use the Auto Function to create tests.

Let’s look at a practical example of code generated by Auto Playwright. Suppose we need to verify the correctness of the user registration process in an application.

To generate an automated test case, compose instructions containing the following directives for the tool:

→ navigate to the registration page;
→ fill in the user details;
→ submit the registration form;
→ verify the results of checks – a success message should be displayed upon registration.

The generated Auto Playwright AI test will look like this:

import { test, expect } from "@playwright/test";

import { auto } from "auto-playwright";



test('User registration', async ({ page }) => {

 await auto('go to the registration page', { page });

 await auto("fill in name, email, and password", { page, test });

 await auto('submit the registration form', { page });

 const registrationSuccess = await auto('is the registration successful?', { 

    page, 

    message: 'Registration successful' 

  });



expect(registrationSuccess).toBe(true);

});

Learn more about the tool by watching the Execute Automation Academy by Karthik K.K video: Auto-Playwright – Run Playwright Test with AI (Using GPT 3, GPT 4 Turbo)

Key features of Auto Playwright:

• Automatic waiting. The tool automatically waits for elements to be ready for interaction, reducing the number of flaky tests.
• Cross-browser testing support. You can run tests in Chromium, Firefox, and WebKit to ensure your solution works consistently across different browsers.
• Code reduction. Auto Playwright streamlines page interactions and reduces repetitive code. For instance, interacting with a registration form field only takes a single line of code instead of multiple lines. To demonstrate, let’s refer to the test from the previous example:

 await auto("fill in name, email, and password", { page, test });

• Automatic element visibility checks. You do not need to write specific assertions for visibility, reducing the complexity of test cases.
• Retry on failures. The tool retries test executions in case of common errors (e.g., browser load failure, element timeout, etc.). This makes tests more reliable and stable.
• Easy navigation. Auto Playwright supports navigation between different pages, tabs, or browsers.

Advantages of Auto Playwright:

• Cost savings. Auto Playwright is an open-source framework, meaning you can use this effective testing solution completely free of charge.
• Improved collaboration on the project. With this Playwright AI tool, you can create readable, concise autotests. This simplifies the work for less experienced testers and allows non-technical specialists to participate in testing.
• Reduced risk of errors. This is facilitated by the advanced features of the Auto Playwright AI tool, including auto-waiting, element visibility checks, retries, and more.
• Wide integration capabilities with testing frameworks. You can leverage Jest, Mocha, Jasmine, or other platforms while benefiting from the power of AI with Auto Playwright.

Comparison of ZeroStep & Auto Playwright

To help you choose the optimal tool for your project, here’s a comparison of ZeroStep and Auto Playwright in a clear tabular format.

What Are the Limitations of Tools for Playwright AI Testing?

In this article, we’ve provided plenty of evidence that artificial intelligence is the future of test automation. However, it’s crucial to consider the limitations that might emerge when using AI-based testing tools. Let’s take a look at the potential challenges teams may face when deciding to implement AI in their projects.

Limitations of ZeroStep:

• Limited User Capabilities. The tool uses a codeless approach to writing test cases. As a result, it may be difficult to automate complex and non-standard workflows.
• Scalability Challenges. The tool is primarily designed for small and medium-sized projects, and managing large test sets can present challenges.
• Basic Integration Capabilities. Users may experience difficulties integrating ZeroStep into their workflows.

Limitations of Auto Playwright:

• Steep learning curve and need for coding skills. This is a code-based tool that requires knowledge of JavaScript or TypeScript. It will also take time to learn if you’re not familiar with Playwright.
• Manual setup. Before starting testing, users must manually configure the testing environment. This can be a complex task.
• Limited test management capabilities. To manage test cases and generate advanced reports, integration with third-party tools is required.

Both tools are powerful solutions for web application test automation. They allow teams to adapt to the dynamic nature of web development and demonstrate excellent results. When choosing a platform, consider the limitations of each. This will help meet your specific testing needs.

Bottom Line

Playwright with AI emerged in response to the need for reliable and efficient web application testing by QA teams. This approach combines the reliable functionality of the Playwright automation library with the advanced capabilities of artificial intelligence.

Testers interested in the rapid creation of tests can use the built-in Playwright Codegen, optimizing its capabilities with ChatGPT and other similar systems. Additionally, third-party AI-based tools are at their disposal.

— Which option to choose We hope our article has helped you make a decision. If you still have questions, reach out to a testomat.io manager for expert consultation on any aspect of testing.

The post Playwright AI Revolution in Test Automation appeared first on testomat.io.

“Why you need Cucumber for API testing?”

What is API testing?

Typically, API (Application Programming Interface) testing focuses on analyzing the app’s business logic, security, and data responses and is generally performed by making requests to one or more API endpoints and comparing the responses with expected results.

Performed manually or with automated testing tools, application program interface or API testing is used to check the API’s functionality and security levels to prevent a negative impact on its performance and reliability.

Why do we need API testing?

When using various websites, apps, and other software products, users want them to be secure, reliable, and free of errors. While there are any issues at the application programming interface layer, it can result in user-facing errors, slow performance, and damage your reputation. Here is why API testing can be useful:

• With application programming interface tests, QA teams can define how different software components interact and check if they work correctly.
• With application programming interface testing, teams can identify potential problems and make sure that the software will handle diverse conditions effectively.
• When it comes to potential software vulnerabilities, APIs can become a potential entry point for attackers. With application programming interface testing, testers can detect weaknesses in software to protect security and privacy.
• If any changes have been made, API testing prevents disruption to existing applications that use this API and helps maintain backward compatibility.

Types of Bugs Detected with Application Programming Interface Testing

Generally, during application programming interface testing we check the following:

• Data accuracy to verify that the data returned from an application programming interface aligns with expected formats, data types, etc.
• Missing or duplicate functionality to make sure that there are no repeated or missed features.
• Authorization or permission checks to verify the identity of a user/application that is requesting access to an API.
• Response time to reveal the duration of time required to process a request and return a response to the client
• Error codes in application programming interface responses to reveal whether the API request was successful or not.
• Reliability, performance, and security issues (checking how well the software performs, finding security problems, or issues like crashes and memory leaks).

Types of API Testing

Here are some of the common API testing examples with different types of tests based on which test cases can be created:

• Functional testing. With functional testing for APIs, you send requests to an API and validate that the correct responses are returned.
• Performance testing. With this type, you can verify that the application programming interface can handle large volumes of data and high traffic.
• Unit testing. As APIs have lots of small moving components, it’s effective to test them during the development phase and validate that each software module performs as designed.
• Integration testing. With integration testing, you can check if your application programming interface works well with different parts of your software, including web services, databases, or third-party applications. But you need to remember that opting for this type is essential if your application programming interface is complicated and not isolated from other components.
• Regression Testing. With this type, you can make sure that your application programming interface is still functional after an update or bug fixing has been performed.
• Security testing. This type of testing protects your API from data breaches, unauthorized access, and other security issues by systematically assessing the application programming interface for potential vulnerabilities and implementing measures to prevent or mitigate them.
• Load Testing. This type of testing helps test an API’s performance under high-user traffic to prevent slow response times, increased server load, and poor user experience.
• Stress Testing. With this testing type, you need to test an application programming interface with large data or sudden spikes in user requests.
• API Documentation Testing. With API document testing, testers make sure that application programming interface documentation is always up-to-date and correct while the API is set up and functions as documented.

What is Cucumber?

Written in Ruby, Cucumber is a BDD framework that allows teams to use a human-readable Gherkin syntax for writing test cases. When test scenarios are written in plain language, teammates without technical backgrounds can better communicate and easily understand what is going on in the project. Cucumber tests are linked to step definitions written in a programming language, which are then executed to verify the app’s behavior.

When using the Cucumber framework for API test automation, you can efficiently test the APIs to uncover any bugs or vulnerabilities before they become problems as well as save time and money in the long run.

Why use Cucumber framework for API testing?

• With Cucumber, teams can write test cases in plain English and avoid misunderstandings about what the tests are verifying.
• Cucumber can be integrated with various tools for application programming interface testing.
• Supporting BDD, teams can focus on the behavior of the application rather than the implementation details.
• Teams can reuse step definitions across different scenarios and maintain tests more easily.
• Thanks to Cucumber, teams can see what has been tested and what the results are.

API Testing Cucumber Your Steps to Follow

With API testing, teams can validate the business logic, data responses, and performance of server-side endpoints. When using the Cucumber BDD framework, testing teams can create tests written in Gherkin and make them understandable for the whole team. However, teams should understand the application programming interface specifications before writing tests. They need to overview the API’s purpose, functionality, endpoints, request methods, expected request formats, and response structures at the very start.

How to set up Playwright & Cucumber?

Prerequisites

On the begining you need to make sure you have the following in place:

A development environment with your preferred programming language (e.g., JavaScript, TypeScript, Python) installed.

node -v
npm -v

Step 1: Start a Project

From the very start, you need to initialize the Playwright project, which can be done by running the next command:

npm init playwright@latest

After that choose the programming language as Javascript when asked in the command line.

Next, you need to add project dependencies and establish configurations for test runners. Install Cucumber and REST Assured dependencies for application programming test interface.

npm i @cucumber/cucumber

Also, add the Cucumber extension to your VSCode:

Finally, you need to install the node-fetch library to your project using:

npm install node-fetch

Anyway, you can use any other HTTP library.

Step 2: Create Feature File

After you start the project and establish the appropriate configuration, it is imperative to create a structure so that your the feature files and step definitions will be well-organized for better clarity and understanding. Cucumber tests start with feature files, which describe the behavior of your application and when creating a feature file, you need to use Gherkin human-readable format with the following structure:

• Feature: it gives explanation to the feature that is going being tested.
• Scenario: it provides information about test scenario.
• Given: it describes the starting condition.
• When: it defines conditions.
• Then: it determines the expected outcome.

Here’s a Cucumber API testing example of a Feature file for the JS project:

Feature: Login User

  Scenario: Successful user login
    Given I send a POST request to the user endpoint with the following payload:
      | email                | password          |
      | jolir86609@ikaid.com | pass              |
    When the response status code should be 201
    Then the response body should contain "John Doe"

We have to create a feature folder and add our file to insert this code. After that add the path to this file and the steps file in vs.code =>settings.json Refer to the screenshot below for guidance:

Create a cucumber.json file and also add a path. It should look like this:

Step 3: Create Step Definition Directory

At this step, you can create a directory called step_definitions and add a JavaScript file (for example, login.js) Similarly, look at the structure of the file tree on top.

const { Given, When, Then } = require("@cucumber/cucumber");
const fetch = require("node-fetch");

Given("providing valid url", async function () {
    this.res = await fetch("https://app.testomat.io/api/login/", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({
            email: "jolir86609@ikaid.com",
            password: "pass",
        }),
    });
});

When("the response status code should be 201", async function () {
    const status = this.res.status; // fetch returns status as a property
    if (status !== 201) {
        throw new Error(`Expected status code 201 but got ${status}`);
    }
});

Then("the response body should contain John Doe", async function () {
    const body = await this.res.json(); // fetch returns JSON via res.json()
    console.log(body);
    if (!JSON.stringify(body).includes("John Doe")) {
        throw new Error(`Response body does not contain 'John Doe'`);
    }
});

In the created file, you can implement the logic for each step defined in your feature file. Generally, step definitions are used to interpret Gherkin’s steps, execute actions like making API requests, and use assertions to validate API responses.

Step 4: Execute Tests

At this step, you can run your tests using the following command:

npx cucumber-js features

With this command, you can execute the tests defined in your Cucumber feature file and step definitions.

My test failed as the email is wrong, I know it. In addition, I checked this POST request with Postman also, you can see it better now.

Step 5: View Test Reports

To generate reports, you can integrate Cucumber’s built-in reporting features or use third-party tools like cucumber-html-reporter Based on test reports, you can get information about test coverage, success/failure rates, and potential areas of improvement.

Pay attention to the “format” settings in your cucumber.json file.

{
  "default": {
    "formateOptions":{
      "snippentInterface": "async-await"
    },
    "paths": [
      "tests/features/*.feature"
    ],
    "publishQuite": true,
    "dryRun": false,
    "require": [
      "tests/steps/*.js"
    ],
    "format": [
      "progress-bar",
      "html: cucumber-report.html"
    ]

  }
}

Cucumber Playwright Reporting with modern test management

Also, you can integrate test case management systems like testomat.io to get access to test results, metrics, and trends and quickly address any issues that arise during testing.

Push the Import button and just copy and paste the command in your CMD offered by TCMS to download your automated tests. By the way, it is a perfect option to sync your manual and automated tests in one place, consolidate Steps:

Similarly, with UI tips our test management suggests – launch a test execution of your Cucumber API Playwright framework.

Do not forget the recommendation to add a Playwright code runner plugin code to playwright.config.js file:

reporter: [
  ['list'],
  [
    '@Testomat.io/reporter/lib/adapter/playwright.js',
    {
      apiKey: process.env.TESTOMATIO,
    },
  ],
];

As a result, you can see such rich test result Report. It provides detailed insights into your test execution and comprehensive test run summaries, it ensures you have a clear view of your testing progress and quality metrics.

Best Practices for API Testing with Cucumber

• You need to remember that grouping your test cases by functionality is important. It will give you a better overview of the app status and improve maintainability.
• You can use mocking and stubbing techniques to simulate interactions with the application programming interface to isolate the application code from external dependencies and make the API testing process more efficient.
• You can apply hooks in Cucumber to set up preconditions and clean up post-conditions for test scenarios.
• You need to automate only what is not possible to be tested via unit or integration tests to avoid additional costs.

Bottom Line: What about using Cucumber test automation framework for Application Programming Interface (API)?

When you develop API-driven applications, you can use Cucumber for API testing. It offers a structured approach for writing human-readable feature files and linking them to step definitions. It helps teams create tests that are easy to understand and maintain.

There is no doubt that API testing is a critical part of the software development process. With Cucumber, you can make it more efficient and effective. By opting for Cucumber, you can deliver high-quality APIs and make sure that different app components function correctly.

If you have any questions, do not hesitate to contact our specialists for consultation!

The post Cucumber framework for API testing with Playwright example appeared first on testomat.io.