cypress Archives - testomat.io https://testomat.io/tag/cypress/ AI Test Management System For Automated Tests Thu, 17 Jul 2025 09:30:21 +0000 en-US hourly 1 https://wordpress.org/?v=6.8.2 https://testomat.io/wp-content/uploads/2022/03/testomatio.png cypress Archives - testomat.io https://testomat.io/tag/cypress/ 32 32 Playwright VS Selenium VS Cypress: A Detailed Comparison https://testomat.io/blog/playwright-vs-selenium-vs-cypress-a-detailed-comparison/ Fri, 07 Feb 2025 15:29:16 +0000 https://testomat.io/?p=17949 The world of test automation offers a multitude of frameworks, making it challenging to select the best one for your product. Selenium has long been established as one of the most popular tools, but newer frameworks like Cypress and Playwright are offering strong competition. The choice between these often depends on the project’s specifics and […]

The post Playwright VS Selenium VS Cypress: A Detailed Comparison appeared first on testomat.io.

]]>
The world of test automation offers a multitude of frameworks, making it challenging to select the best one for your product. Selenium has long been established as one of the most popular tools, but newer frameworks like Cypress and Playwright are offering strong competition.

The choice between these often depends on the project’s specifics and the developer’s preferred programming language. Syntax and ease of use play an important role: some find Selenium’s approach more intuitive, while others prefer Playwright.

Each tool has its advantages and drawbacks in software development. Switching from Selenium to Cypress reveals its speed compared to the former. Test scenarios that once consumed significant time and frequently encountered errors now execute quickly and seamlessly, both locally and in CI\CD pipelines. Playwright, a relatively new tool, is an open-source framework developed by Microsoft for end-to-end testing and is currently the fastest available option.

This article provides a closer look at each of these frameworks and compares them. The information will help you choose the best tool for your project. For an introductory overview, check out the video.

What is Playwright?

Playwright supports most modern rendering browsers, such as Mozilla Firefox, Chromium, and WebKit. Its popularity is steadily increasing, enabling fast, efficient, and reliable cross-browser testing.

This tool is a Node library that automates browsers through a unified API.

The Playwright framework supports popular continuous integration servers like Jest, Mocha, and Jasmine, while also offering cross-language support, including TypeScript, JavaScript, Python, .NET, and Java.

Key Features of Playwright:

  • Universal API for any web browser and platform. It operates on Windows, Linux, and macOS and supports native mobile emulation for Google Chrome on Android and Safari on iOS.
  • Reduced test flakiness. Playwright automatically waits for interactive elements to become available before performing actions. It includes features like retries, trace recording, video recording, and screenshots.
  • Isolated test execution. Each test runs in a separate browser context, equivalent to creating a new web browser profile without additional resource costs.
  • Powerful built-in tools. Codegen allows recording test cases and saving them in any language (JavaScript, Python, .NET, Java, TypeScript). Playwright Inspector enables inspecting the page, viewing click points, and analyzing execution logs. Trace Viewer helps capture all the details necessary for analyzing test failures.

Playwright is a top-nonch solution for end-to-end testing that continues to evolve and improve.

Playwright Architecture

Playwright interacts directly through the WebSocket protocol. During test execution, the code is converted into JSON format and transmitted to the server using a WebSocket connection for command exchange. This connection remains active until one of the sides terminates it.

Illustrated WebSocket Benefits

💨 One of the reasons behind Playwright’s high speed is its ability to maintain an active connection for an extended period.

What is Selenium?

Selenium is an open-source framework designed for automated testing of web applications across different platforms and browsers. It supports test automation using programming languages such as Java, JavaScript, C#, Ruby, PHP, and Python.

The framework allows testers to utilize a network of browser instances for parallel execution of test scripts, significantly accelerating the testing process and the overall development cycle.

Selenium Infrastructure

Selenium Tool Suite

  1. Selenium WebDriver is an API library that is invoked from a script to execute commands in a chosen browser. Using Selenium locators, it is possible to find and define elements for creating test scripts and interact with them. For each browser, there are specific drivers that interpret the scripts written for them.
  1. Selenium IDE is a user-friendly tool for recording, editing, and executing test scripts. It is available as an extension for Mozilla Firefox, Google Chrome, and Microsoft Edge. It enables quick creation and execution of automated tests without writing code by recording browser actions such as clicking elements or filling out forms and replaying them to verify the functionality of a web application. Additionally, IDE allows editing scripts, exporting them into various formats (e.g., HTML, Java, Python, or C#), and adding assertions to validate application behavior.
  1. Selenium Grid is a tool for executing tests on various browsers, machines, and operating systems This functionality is especially useful for large test suites that need to be executed as quickly as possible.

In Selenium Grid, there is a central hub that manages the connected nodes. For example, one node might run Google Chrome on Windows, another Mozilla Firefox on macOS, and yet another Safari on iOS. The hub serves as the central point coordinating test execution across nodes. Nodes are the machines that execute tests. Once the nodes connect to the hub, a link is established between them. Tests are written using client libraries like Selenium WebDriver. Commands are sent to the hub, which redirects them to the appropriate nodes for execution.

By leveraging Selenium Grid, test execution can be significantly sped up by running them simultaneously on multiple machines and browsers.

Selenium Architecture

Starting with Selenium 4, the JSON Wire Protocol was replaced by the W3C WebDriver protocol, which is now the official standard for managing web browsers. Despite this, the Selenium team continues to support the legacy protocol, which can still be used via language bindings and the server.

The new protocol, known as “WebDriver W3C,” has been approved by the World Wide Web Consortium (W3C). In Selenium 4’s new architecture, the client and server communicate directly, eliminating the need for the JSON Wire Protocol. This has resulted in faster test script execution and significantly reduced instability.

Selenium Client Libraries

The W3C WebDriver protocol offers several advantages. Compared to the JSON Wire Protocol, it significantly expands capabilities. For example, the Actions API has been enhanced according to the WebDriver specification. It now allows actions such as zooming in/out and simultaneously pressing two keys with multi-touch gestures.

What is Cypress?

Cypress is an interface testing tool designed for modern web applications. It supports JavaScript and TypeScript and is used for end-to-end testing with the Mocha framework, simplifying asynchronous testing. Some time ago Cypress was a popular and promising solution considered as alternative to Selenium by many companies.

Key Advantages of Cypress:

  • Ease of installation. Getting started requires only installing it via the npm or yarn packages or using the desktop downloading. There is no need to install additional libraries, dependencies, or drivers, as everything needed comes bundled with Cypress.
  • Test execution speed. Cypress runs the test suite directly in the browser, and its learning curve is significantly faster than Selenium. In addition, with the built-in Chai library supporting Behavior-Driven Development (BDD) syntax, creating test scripts is quick and straightforward.
  • Real-time automatic updates. After making changes to a test script, Cypress automatically reruns and updates the tests without the need for manual reloading.
  • Accelerated test writing. In just a few minutes, you can begin creating test scripts, as Cypress delivers results that closely simulate real user experience.
  • Automatic waiting. There’s no need to manually add waits or delays in tests. Cypress automatically waits for commands and assertions to complete before proceeding to the next step.
  • Screenshots and videos. During test execution, videos of the entire test suite are recorded automatically, along with screenshots in case of failures when running headlessly. This greatly simplifies error analysis.
  • User-friendly Cypress dashboard. Using the Cypress Cloud service, you can run tests in CI\CD environments and receive detailed reports on their execution.

Cypress addresses key challenges developers and testers face when working with modern applications, making the testing process fast, efficient, and convenient.

Cypress Architecture

Cypress is a test automation framework for web applications that stands out due to its unique architecture, designed to accelerate and enhance the reliability of testing. User interface testing is conducted directly in the browser, without the need for dependency on driver binary files. This eliminates network delays and speeds up the process.

Cypress architecture
How Cypress interacts visualization

Key Features of how Cypress Architecture works

Cypress operates using a Node server that constantly interacts with the browser via WebSocket, synchronizing processes and performing necessary tasks. This enables easy access to elements:

  • DOM (Document Object Model)
  • Local storage
  • Network layer
  • Window object

Such mechanism is a significant advantage of Cypress, making it a quite convenient tool for developers and testers.

Comparing Playwright vs Selenium vs Cypress Features

You can use a single tool, but it is better to utilize multiple tools simultaneously for maximum project coverage.
Sergey Almyashev
COO, ZappleTech Inc

Let’s analyze how essential functions are implemented in each of the compared frameworks.

Installation & Configuration

The setup process for test automation tools can vary significantly in terms of complexity. Selenium, Cypress, and Playwright each have unique installation requirements tailored to different testing needs. Here’s how to prepare each tool for use, so you can choose the best one based on your project requirements.

Playwright

Playwright is known for its simplicity in installation and configuration:

  • Ensure Node.js is installed and its version is the latest.
  • Set up a code editor, such as Visual Studio Code.
  • Choose an installation method.
  • VS Code extension. Create a folder, open it in the editor, find and install the Playwright extension. After that, the installation process will start automatically.
  • Init command. Create a folder, open it in the editor, and run the command
    npm init playwright@latest

    During installation, select the project language (TypeScript or JavaScript).

After setup, you will have a ready-to-use .spec file.

Cypress

It is easy to set up as well, making the installation process as convenient as possible. Thanks to the structured organization of folders, you can easily create the foundation for your project.

To install Cypress, follow these steps:

  1. Check if Node is installed on your computer.
  2. Create a json file using the npm initcommand.
  3. Install the latest version of Cypress by running
    npm install cypress --save-dev
  4. Check that the json file reflects the changes.

After launching Cypress, you will immediately have access to the default .spec files for quick test initiation.

Selenium

Setting up Selenium can be a challenging task for beginners in testing! The installation process typically involves multiple steps.

    1. To begin, install Selenium geckodriver After downloading, extract the ZIP file to your chosen folder. Place it in /usr/bin or /usr/local/bin
    2. The next, install WebDriver. Download the latest version of Selenium WebDriver for your preferred programming language from this link For example, apply the step for Python using pip package:
      pip install -U selenium

Runner

The test runner plays a key role in running tests and exporting their results. As a library, it scans the source code directory, finds and runs test files to detect errors and failures. The runner also ensures the correct execution of tests, providing a report on their compliance with the set standards. It can run functional tests, generating results according to your configuration.

It is important to distinguish between test automation software, test run orchestrators, and execution environments.

Playwright runner

Playwright has its own test runner and also supports integration with other popular runners and works with multiple major browsers using environment variables. By default, headless testing is performed, and the results are displayed directly in the terminal. To connect Playwright to your existing test runner, you only need a lines.

Main Playwright commands:
npx playwright test

npx playwright test fileName.spec.js //Run a single test
npx playwright test tests/page/home-page/  //Run a test suite
npx playwright test home-page.spec.js --headed //Run tests in headed mode
pytest 
pytest test_login.py #Run file
pytest tests/test_todo_page.py tests/test_landing_page.py #Run specific test

Playwright supports debugging mode, which allows detailed analysis of test execution using the Playwright Inspector tool. This tool allows you to:

→ Step through Playwright API calls
→ View debug logs
→ Analyzing locators by hovering the cursor over an element during inspection

During debugging, Playwright Inspector opens a convenient interface where you can step through and examine the reasons for test failures and fix them directly during the analysis process.

Selenium runner

To execute tests using Selenium, you can use:

  • Selenium WebDriver – the core component of Selenium that allows automation of actions in web browsers, such as clicks, form filling, and navigation.
  • TestNG – a popular testing framework often used with Selenium. It supports test scenario creation, test suite organization, and parallel execution.
  • JUnit – another widely used testing framework for Selenium with similar capabilities to TestNG, including test case creation and parallel execution.

For test execution orchestration, you can use:

  • Jenkins – a continuous integration tool that automates the build, testing, and deployment processes. It integrates with Selenium, supporting parallel testing and optimizing the software delivery process.
  • Maven – a powerful build automation tool that simplifies managing Selenium tests, defining dependencies, automating testing processes, and deployment.

For test execution, you can use Selenium Grid or a local Selenium server. Selenium Grid allows you to run scripts on remote machines or in cloud environments, enabling parallel test execution across different browser and operating system configurations.

Cypress

The Cypress Runner provides a convenient graphical interface for visualizing and debugging tests. It features a test list panel, an execution window displaying commands and their results, and a browser showing the real-time application being tested.

Its debugging tools include:

→ Console for logging messages and interacting with the application in real-time
→ Tab for monitoring HTTP requests and responses.

This Runner provides instant behavior analysis of the application, helping to quickly identify and resolve issues. Its interactive interface makes it easy to review test results and interact with the application.

The Cypress test status menu provides the following information:
  • Test status. The status of the currently running test is displayed at the top of the panel. The test status can be one of the following: Running, Passed, Failed, Pending, or Canceled.
  • Number of tests. The total number of tests in the test suite is displayed next to the test status.
  • Duration. The total time it took to run the test suite is displayed below the number of tests.
  • Test results. Each individual test is listed below the duration, along with its status (pass or fail) and the time it took to run.
  • Command log. The command log displays a detailed list of all the Cypress commands that were executed during the test run.

The test status menu provides a quick and easy way to monitor the progress of your Cypress test suite and to identify any failing tests or errors that need to be addressed.

Orchestration of test runs:
  • GitHub Actions – a tool for automating Cypress tests in a CI\CD pipeline that integrates with GitHub repositories and allows tests to run on every push or pull request.
  • CircleCI – a CI\CD platform supporting parallel test execution, which speeds up the testing process by distributing tests across multiple machines.
  • Jenkins – can also be used as an orchestrator for running Cypress tests.

Language/Browser Support

Framework Languages Browser
Selenium Java, Python, C#, Ruby, JavaScript. Google Chrome, Mozilla Firefox, Microsoft Edge, Safari
Cypress JavaScript, TypeScript Google Chrome, Mozilla Firefox, Microsoft Edge, Safari
Playwright Python, Java, JavaScript, .Net Google Chrome, Microsoft Edge, WebKit, Mozilla Firefox

Record & Playback features

All three tools support record and playback functionality.

Playwright

In Playwright, test scripts are created by writing code that simulates user actions. To simplify this process, the CodeGen tool is used, which automatically generates code based on browser interactions. It records each user action, creating ready-to-use test commands, significantly speeding up test development.

Selenium

Selenium IDE allows easy recording and playback of test scripts. To use it, simply install the plugin in the browser — no additional setup is required. Unlike Selenium WebDriver and RC, Selenium IDE does not require programming code for creating test scripts. Instead, you can record browser interactions, which are automatically converted into test cases. These scripts can then be executed using the playback feature.

Cypress

Cypress Studio is a visual tool that allows you to create test cases by recording interactions with the application under test during real-time interaction with the app’s interface.

Cypress Studio automatically generates test code based on interactions with DOM elements and records them.

Element Selection

Element selection is the process of filtering all elements with a specific tag in documents and verifying whether the element you are working with is detached from the DOM.

Playwright

In Playwright, the concept of automatic waiting is implemented, which allows test execution to continue after selecting an element. Playwright automatically waits for all necessary checks to be completed. After successful completion of these checks, the requested action is performed.

Selenium

Web elements in Selenium are used to interact with DOM elements. Different types of locators are usually used to connect to page elements. Sometimes, when searching for an element, we assume it is already loaded on the page. However, this can lead to test failures if the element has not been loaded yet. To avoid this issue, we use implicit or explicit waits.

To prevent test failure, we check for the presence of an element in the DOM:

Selenium Java example:
WebElement button = new WebDriverWait(driver, 30).until(ExpectedConditions.presenceOfElementLocated(By.id("buttonId")));

button.click();

Cypress

In Cypress, the opposite occurs, as interaction with elements is done through locators. It has a built-in waiting mechanism that makes test execution more stable and reduces the likelihood of errors.

This waiting time is configurable using the defaultCommandTimeout option, which is set to 4000 ms by default. If an element is not loaded within this time, an error message appears. The value can be changed in the cypress.config.js file:

Cypress JavaScript example:
const {defineConfig} = require('cypress');

module.exports = defineConfig({

defaultCommandTimeout: 10000

});

Cypress automatically pauses for network requests to complete before moving on to the next command. In the examples below, Cypress waits 4 seconds for each command:

cy.get('.btn').click(); // Click the button

cy.focused().click(); // Click on the element with focus

cy.contains('Welcome').click(); // Click on the first element containing 'Welcome'

Parallel Execution

Parallel execution is an important feature for web browser automation, as it allows multiple test cases to run simultaneously, significantly reducing overall execution time. Selenium, Cypress, and Playwright are leading tools for parallel execution that have gained popularity among developers.

Playwright

Supports parallel test execution across multiple machines. To do this, specify the number of workers via the command line, for example:

npx playwright test --workers 4

Alternatively, modify the configuration file playwright.config.js.

Selenium

Parallel execution in Selenium means running tests from different modules simultaneously rather than sequentially. This can be achieved using TestNG. Here is an example of a configuration in TestNG XML for parallel execution:

xml

<suite name="ParallelTesting" parallel="methods" thread-count="3">

In this case, you can configure parameters such as methods, instances, tests, or classes for parallel execution. thread-count="3" ensures both test classes of our 3 tests run in parallel.

Cypress

Supports parallel execution starting from version 3.1.0. By default, it runs tests serially, but for parallel execution, you can connect Cypress Cloud to CI\CD and use the command:

C# Cypress example:
npx cypress run --record --key <your-key>

Flakiness

Flaky tests give unpredictable results, sometimes passing and sometimes failing, even when the code being tested hasn’t changed. This inconsistency creates difficulties for development teams, unlike stable tests that always produce the same results.

Playwright

The framework offers various strategies to solve test instability issues:

  • Automatic waiting. Use automatic waiting to ensure actions (e.g., click or data entry) occur only after the element has fully loaded.
  • Using waitFor methods. Use like waitForLoadState,waitForNavigation, and waitForSelector, to reduce instability by ensuring the page is fully loaded before interacting with elements.
  • Retrying tests. Use the retry feature for cases when required elements might not be available at the initial stage of page load, causing test failures. By configuring retries in config.js the system automatically re-runs failed tests, increasing their chances of success.

Selenium

Test instability in Selenium can arise from issues with test code or test infrastructure.

To reduce instability, use the following strategies:

  • Use stable selectors. For example, use classes or IDs instead of dynamic ones like XPath, as the latter are sensitive to changes in the HTML structure, which can lead to instability.
  • Implement explicit waits. Replace implicit waits with explicit waits to ensure the page is fully loaded and stabilized before interacting with elements. This will help avoid test failures due to elements not being immediately available.
  • Add retry mechanisms. This will retry failed checks a specified number of times before stopping. It helps reduce instability caused by temporary network or server issues.
  • Ensure a stable test environment. Network and server conditions must be consistent to minimize external factors that could cause instability.
  • Debug test failures. Spend time understanding the reasons for test failures. Identifying and addressing the root causes can effectively reduce test instability.
  • Use reliable testing frameworks. For example, JUnit or TestNG, to organize tests and effectively manage failures. These provide built-in support for handling test failures, simplifying the debugging process.
  • Rely on a reliable CI server. For example, Jenkins or Azure DevOps. They automate test execution and systematically monitor results.

Cypress

Cypress effectively handles test instability and offers various methods to address this issue:

  • Assertion commands. Use checks such as .contains('element'), .should('be.visible') and .should('exist')
    before interacting with elements. This enhances the reliability of tests by verifying the presence and visibility of elements before performing actions.
  • Test retries. You can configure test retries globally, for each test, or for test sets to automatically retry failed tests and increase the likelihood of their success.
  • Timeouts instead of waits. Suitable for situations where elements require more time to load. By configuring timeouts at the field level, Cypress will continue the next command if the element loads before the specified timeout, improving test efficiency.
  • intercept() command. Use this to wait for XHR requests to complete. By intercepting specific network requests and waiting for their execution, you can avoid test failures caused by incomplete requests.

Screenshots

Screenshots are an essential tool for diagnosing frequently failing tests, as they help identify the stages where errors occur and assist in improving tests to reduce instability.

Playwright

Offers three methods for capturing screenshots:

  • for the entire page
  • saving to clipboard
  • capturing specific elements

This provides comprehensive visual documentation of test execution, facilitating effective debugging and enhancement of test scenarios.

Selenium

In Selenium WebDriver, capturing screenshots is indispensable for identifying errors. During test execution, the TakesScreenshot method can be used to instruct WebDriver to take a screenshot.

Cypress

Has a built-in function for taking screenshots. During test execution through cypress open or cypress run you can manually take a screenshot using cy.screenshot() To disable automatic screenshot capture on test failure, you can set the screenshotOnRunFailure parameter to false in the Cypress configuration or in Cypress.Screenshot.defaults(). This provides flexibility in managing the screenshot process according to testing needs.

Video

Video recording features are crucial for identifying errors and simplifying the debugging process, especially when tests fail. Visual recording of the sequence of test steps allows easy identification of the exact point where failures occur.

Playwright

Test video recording is managed through the video parameter in the configuration file. By default, video recording is off.

When configuring the video parameter, the following options can be selected:

  • off – disables video recording.
  • on – records video for each test.
  • retain-on-failure – records video for each test, but deletes the video for successful tests.
  • on-first-retry – records video only on the first retry of a test execution.

These options allow configuring error recording according to specific testing requirements and preferences.

Selenium

Selenium WebDriver does not have a built-in feature for recording video during test execution. However, there are extensions that allow recording all steps. These extensions help precisely identify the cause of failures at specific stages of the test. For instance, it might be the FFmpeg library. 

Cypress

Automatically records video for each .spec file. During the cypress run command, the video is recorded and saved in the folder cypress/videos path. This feature can be fully disabled by setting the video parameter to false in the configuration file.

Iframes

Iframes are HTML documents embedded within other HTML documents, typically used to add external content, such as YouTube videos or images, to web pages. If the iframe belongs to the same domain as the originating page, it can be automated using browser automation.

Playwright

To test iframes, the FrameLocator method is used, which allows access to elements within the iframe. This method handles the access logic, ensuring effective interaction.

Selenium

Provides a simple way to work with iframes. Users need to switch to the iframe window to perform the required actions and then return to the main page for further interactions. By default, Selenium works with the primary browser driver, so the driver’s focus must be changed from the main window to the specified frame element.

There are several ways to switch between frames in Selenium:

  • switchTo().frame(id)– specify the ID or name of the frame as an input parameter for the switchTo()
  • switchTo().frame(i)– specify the index of the frame as an argument (indexing starts from zero).
  • switchTo().frame(webelement n)– pass the web element of the frame as an argument.
  • switchTo().defaultContent()– returns focus from the iframe to the main page.

Cypress

Direct interaction with iframes is not supported. A plugin must be installed, and then special commands can be used to interact with elements inside the iframe. Once installed, custom commands can be used for effective interaction.

Handling API Requests

Testing includes verifying data related to elements on the API (Application Programming Interface) side – an intermediary layer between the presentation and data layers that facilitates interaction between software applications. Consequently, API testing has become an integral part of quality assurance.

Playwright

Provides access to the application’s REST API, allowing automation of requests using GET, POST, PUT, and DELETE methods.

Pay attention, we are considering Playwright API testing with a few articles on our blog already:

Selenium

Primarily used for E2E UI automation and is not intended for API testing. However, through third-party tools, it is possible to partially automate API testing using Selenium Grid. This is done by opening the browser, navigating to the API endpoint, and sending HTTP requests to verify the response.

Cypress

Allows automation of API testing through thecy.request()command, which supports GET, POST, PUT, and DELETE methods. Read another article on our site — Cypress and REST API Testing

Community Support

Each framework has its own community, which helps users understand the tools and receive support in resolving issues during work.

Playwright

This is a relatively new tool on the market, so its community is still small but growing quickly. Numerous resources are available to help users get started with Playwright. The team offers detailed documentation and actively engages on GitHub, where bugs can be reported, new features requested, or contributions made to development. They respond quickly to questions and community feedback.

Selenium

Has a large and active community of users and developers. Participants have several ways to obtain support and assistance with the tool:

  • Selenium documentation includes tutorials and help guides for installation or setup information.
  • Forums are ideal for discussions and problem-solving. There are separate forums for different programming languages, as well as a general forum for discussions.
  • Stack Overflow is a popular site for programmers, with many questions and answers related to Selenium.

This greatly simplifies the use of Selenium.

Cypress

Has a growing community that provides support through various resources and excellent official documentation. It includes an installation and usage guide, a complete API reference, and a project example library.

In addition, there are many unofficial forums and communities where user interaction occurs (more experienced users) to receive help and share experiences. Many blogs, tutorials, and video materials are available to help learn how to work with Cypress for various types of testing.

In addition to the mentioned channels, there are many user-created resources on platforms like Dev.to or Reddit, where users can get help.

Pros and Cons Selenium VS Cypress VS Playwright

Each automation testing framework has its advantages and disadvantages. Let’s examine them in more detail and in a structured way.

Playwright

It is an ideal choice for teams who value speed and efficiency in testing. Playwright is suitable for modern web applications with a need for cross-browser compatibility.

✅ Advantages ❌ Disadvantages
Easy setup and modern API A relatively new tool with a smaller knowledge base and a small community
Supports multiple programming languages Tests browser engines as opposed to full browsers
Automatic waiting, web-first assertions, and tracing feature Lack of support for native mobile applications
High speed and efficiency Limited browser support
Cross-browser and cross-platform support

Selenium

It is a versatile testing tool that is suitable for large projects and complex scenarios.

✅ Advantages  ❌ Disadvantages
Support for all popular browsers and multiple programming languages Complex setup, which can take a lot of time
Adheres to open standards and governance principles Lack of built-in automatic waiting, which can sometimes lead to unstable tests
Various options for parallel test execution Lower speed due to architectural features
No restrictions on the use of domains, multiple windows, or iframes No built-in reporting feature
Large knowledge base and community support Requires an external runner

Thanks to its flexibility and rich integration capabilities, Selenium will be useful for teams that require cross-browser and cross-platform testing.

Cypress

It is perfect for web applications that require fast and interactive testing.

✅ Advantages ❌ Disadvantages
Simple and intuitive API Supports only Chromium-based browsers
Automatic element waiting and retry actions Limited to JavaScript and TypeScript
Quick debugging capability through instant reloading No built-in support for parallel test execution
Built-in video recording and screenshot features For functions like XPath or reporting, third-party tool integration is required
Detailed documentation and active community support Limited support for cross- browser testing

It will be useful for teams that use JavaScript and TypeScript and need an intuitive tool.

Playwright VS Selenium VS Cypress:

🤔 Which Should Be Used in 2025?

To mitigate the challenges in QA automation testing services, establish a process of continuous monitoring and communication between teams and document actions in the environment. This is a challenging task, but only in cases where you do not have experienced managers in your teams.

Before spending time and effort mastering a testing automation framework, it is important to understand if this tool meets the needs of your project.

✅ Cypress will be an ideal solution for developers working with modern web applications built on JavaScript frameworks such as Vue.js It excels in testing Single-Page Applications (SPAs) and web applications with complex interaction logic.

✅ Selenium allows for the creation of intricate test scenarios, parallel test execution, and even working with minimized browsers. However, its speed is slower than that of Cypress, which may be a notable disadvantage for projects with tight deadlines.

❌ In contrast, Cypress might not be the best choice for developers who require end-to-end (E2E) testing, particularly when dealing with applications containing pop-up windows or iframes. It also is not suitable for performance testing and does not support mobile device emulation, so it is better to choose another tool if these are the main testing goals.

✅ Selenium will be useful if your project involves working with legacy browsers or systems, while Playwright is better suited for modern web applications that use the latest browser technologies. If your team requires support for multiple programming languages and integration with legacy testing platforms or complex CI\CD ecosystems, Selenium is the right choice.

✅ Playwright will serve as the foundation for best practices if you need a quicker and more straightforward process for setting up and running tests. It is ideal for teams that prioritize built-in debugging and testing tools over-relying on third-party solutions. It is perfect for teams that need extensive browser support and advanced features. It will also be beneficial for those working with various programming languages, testing mobile applications, or aiming to ensure parallel test execution.

❌ Although Playwright’s popularity is growing, it may not be ideal for developers who require advanced debugging tools. Furthermore, its API Request method does not support blocking tracking redirects, which makes it less practical for security researchers, ad blocker developers, and similar projects.

As for the speed of test execution across different frameworks on a static site (login verification), individual tests, and test sets, the data is shown in the diagram:

Average test execution time Playwright vs Selenium vs Cypress

The final performance ranking is as follows:

Test execution speed ranking Playwright vs Cypress vs Selenium

Thus, Playwright takes the leading position in real-world scenarios. It consistently demonstrates faster execution in all three cases. In real-world situations, it showed the highest stability (least variability) in execution time.

Cypress has a longer startup time compared to other tools, which affects the speed of executing short tests, although this effect is smaller for longer tests. In longer test sets, Cypress’s speed is comparable to Selenium’s, which is the benchmark for E2E testing.

For now, choose the framework that best aligns with the specific requirements of your project.

Trends in Playwright Usage

Playwright is continuously improving, and in 2025, the following trends are anticipated:

  • Integration with Artificial Intelligence. This will facilitate automatic test creation, result analysis, and the prediction of potential issues, improving testing accuracy and efficiency.
  • Broader support for various platforms. Playwright is expected to expand its compatibility with different operating systems and browsers, including mobile platforms, allowing web application testing across an even larger variety of devices and browsers.
  • Improved speed and performance. Playwright developers are working on optimizing test speed and reducing setup time, which will accelerate test execution.
  • Easier integration with CI\CD. This will automate the testing process and significantly speed up defect detection.
  • Enhanced documentation and community support. This should simplify the tool’s learning curve for new users.

Today, Playwright will keep incorporating cutting-edge technologies and improvements to maintain its dominant position among automated web application testing tools.

Summary

In modern programming, test automation is often achieved using tools such as Selenium, Cypress, and Playwright. Each of these tools has unique advantages and limitations, making them suitable for various testing needs.

Selenium, being one of the oldest and most widely recognized frameworks, provides excellent support for multiple programming languages and browsers, making it essential for end-to-end testing.

Cypress offers a contemporary approach to automation, emphasizing direct browser testing, user-friendliness, and seamless integration for unit testing.

Playwright, on the other hand, demonstrates flexibility with support for multiple browsers, parallel test execution, and the automation of complex tasks like mobile device emulation.

The popularity of Playwright is growing. However, Selenium remains the primary choice for many teams due to its time-tested stability and wide functionality, especially with Java or Python languages.

The choice of test automation tool ultimately depends on the specific needs of the project, the developers’ experience, and the features of the application that needs to be tested.

The post Playwright VS Selenium VS Cypress: A Detailed Comparison appeared first on testomat.io.

]]>
Cypress Code Coverage https://testomat.io/blog/cypress-code-coverage/ Wed, 07 Feb 2024 12:09:20 +0000 https://testomat.io/?p=3782 During project work, testers often have to write many component and E2E tests. However, no matter how many tests are written, questions inevitably arise within the team sooner or later. 💭 Are all sections of the application’s source code executed during testing? 🤔 Are there parts of the web application that are tested too frequently, or […]

The post Cypress Code Coverage appeared first on testomat.io.

]]>
During project work, testers often have to write many component and E2E tests. However, no matter how many tests are written, questions inevitably arise within the team sooner or later.

💭 Are all sections of the application’s source code executed during testing?
🤔 Are there parts of the web application that are tested too frequently, or conversely, not tested at all?

If you’re using Cypress code coverage reports instruments, it can help answer these questions.

What is Code Coverage?

Code Coverage is an indicator that allows testers to understand what portion of your source code is covered by tests. Also it helps evaluate the quality of test sets and, if necessary, make changes to optimize project work.

In Cypress, code coverage is not part of the tool’s out-of-the-box features, so its use involves additional configurations and installing the Cypress code coverage plugin. Let’s explore how code coverage in Cypress can be helpful and what steps are required for its use.

Importance of Collecting Code Coverage

Installing the code coverage plugin allows you access additional Cypress capabilities. Detailed coverage reports allow you to:

  • Determine the feasibility of testing individual web application components. Special metrics are used for this:
    Function coverage shows how many times individual functions were called during testing.
    Statement coverage demonstrates the number of executed statements.
    Branch coverage is a measure of how many branches were successfully executed.
    Condition coverage controls over boolean subexpressions checked for true or false.
    Line coverage counts the number of lines of application code covered by tests.
  • Identify dead code and eliminate it. Dead code refers to source code that can be successfully executed, but its execution results will not affect the program.
  • Save time for QA engineers. Coverage information eliminates the need to search for unchecked or missing test cases.
  • Control the functionality of the code. From the code coverage report, you can obtain information about the parts of the source code and its integrity.

As you can see, code coverage is an essential part of testing. However, in the modern QA world, where there is an increasing emphasis on test automation, automation coverage is another concept to consider. It’s important not to confuse the two.

Differences between Code Coverage and Automation Coverage

Code Coverage and Test Automation Coverage are two components of testing that fundamentally differ from each other. The distinctions between them are outlined in the table below:

  Code Coverage Automation Coverage
Definition Metric indicating what portion of the source code was executed during testing. Indicator allowing assessment of the percentage of test cases executed during automated testing.
Testing Method White-box testing. Black-box testing.
Calculation Basis Source code. Test cases.

In this material, we will specifically discuss Code Coverage for Cypress.

Cypress Code Coverage installation, configuration, options & settings

To use Cypress code coverage, three steps need to be taken:

  1. Instrument the code.
  2. Install the code coverage plugin.
  3. Open the saved code coverage report.

⬇ Let’s delve into each of these steps in more detail!

Setting up Code Coverage

Before proceeding with the plugin installation, ensure that your Cypress project already has a peer dependency installed.

To install the Cypress code coverage plugin, use the following command:

npm install -D @cypress/code-coverage

Then, insert the code into the cypress support e2e.js file and setupNodeEvents function:

// cypress/support/e2e.js
import '@cypress/code-coverage/support'
module.exports = (on, config) => {
on('task', require('@cypress/code-coverage/task'))
}

Conclusion: Specific Considerations for Cypress Usage

Cypress is a tool that supports component and End-to-End tests for web applications. For reliable E2E testing, QA engineers should regularly collect code coverage – an indicator demonstrating how well tests cover the web application’s source code.

You can get this information from the code coverage report, which shows coverage for statements, branches, functions, and lines of source code metrics with Cypress code coverage plugin.

We hope this guide on Cypress code coverage will help you organize quality testing individual functions of your web application.

Have more questions? Contact us and get a detailed consultation on implementing end-to-end and other types of testing for digital solutions.

The post Cypress Code Coverage appeared first on testomat.io.

]]>
How to create readable End-to-End tests with Cypress and Cucumber https://testomat.io/blog/how-to-create-readable-end-to-end-tests-with-cypress-and-cucumber/ Fri, 19 Jan 2024 10:11:02 +0000 https://testomat.io/?p=12262 Delivering reliable and error-free apps is crucial for any company. With end-to-end testing, they can launch robust software products and make sure that all components function as expected. However, development and QA teams often encounter challenges in maintaining and creating readable end-to-end test scripts, especially the non-technical specialists have particular difficulties. That’s where Cypress and […]

The post How to create readable End-to-End tests with Cypress and Cucumber appeared first on testomat.io.

]]>
Delivering reliable and error-free apps is crucial for any company. With end-to-end testing, they can launch robust software products and make sure that all components function as expected. However, development and QA teams often encounter challenges in maintaining and creating readable end-to-end test scripts, especially the non-technical specialists have particular difficulties. That’s where Cypress and Cucumber technologies come to change the game by addressing the challenge.

With Cypress technology, teams can drive rapid execution while Cucumber enables them to employ a natural language format and make any test scenario human-readable.

However, before navigating the essentials of these tools and gaining actionable insights into creating clear, expressive, and maintainable automated tests in the realm of automated testing, we are going to overview some challenges the teams face and discover how to address them through strategic integration of Cypress and Cucumber. Let’s jump in 😊

What challenges do teams face when dealing with E2E tests?

Maintaining and creating readable end-to-end test cases isn’t always easy and there are a few challenges that you should be prepared to face. Here you will learn about some of the hurdles you may face. So, let’s dive right in.

  • Flaky Tests. They pose a challenge in end-to-end testing due to their unpredictable nature such as network issues, browser rendering variations, or external dependencies where the same test may pass or fail inconsistently. This leads to wasting time on debugging false positives.
  • Maintenance Burden. End-to-end test cases often rely on specific UI elements. When any changes to the UI are made, it can break your tests. It requires tedious updates for the team to keep tests functional.
  • Debugging Complexities. When a test fails, it can be challenging to pinpoint the exact reason, particularly when it involves interactions among multiple systems or components. This complexity may result in prolonged debugging sessions and potential blame-shifting across teams.
  • Slow Execution. End-to-end test cases often involve simulating real user interactions that can take a long time to execute. This can slow down your development cycle and make it difficult to get rapid feedback on implemented changes.
  • Data Management Headaches. End-to-end test cases often require specific test data to function properly. Managing and ensuring the consistency of this data across different environments can be complex and time-consuming.

The integration of Cypress and Cucumber technologies is an excellent reason to seriously explore their implementation for creating readable end-to-end test cases.

Cypress and Cucumber: What are they?

Let’s overview Cypress and Cucumber technologies and discover why you need to harness their strengths.

Cucumber

Cucumber is a behavior-driven development (BDD) tool that uses a Gherkin syntax for describing software behaviors in a natural language format. It encourages collaboration between technical and non-technical stakeholders by providing a common language for expressing requirements. With Cucumber, you can do the following:

  • Describe features and scenarios in plain English using Gherkin syntax.
  • Create reusable step definitions in your preferred programming language.
  • Leverage parameterization to efficiently handle different test scenarios within a single feature file.
  • Generate comprehensive and human-readable reports for a quick assessment of test results.
  • Collaborate effectively with stakeholders thanks to Cucumber’s BDD approach that drives comprehension of test scenarios throughout the development process.
  • Integrate Cucumber with diverse testing frameworks.

Cypress

Cypress is a modern end-to-end testing framework designed for web applications. Known for its fast and reliable testing capabilities, Cypress provides also an easy-to-use API for interacting with elements on a webpage if you need it. It offers real-time reloading, automatic waiting, and powerful debugging features, making it a popular choice for testing web applications. With Cypress, you can do the following:

  • Perform cross-browser testing thanks to Cypress’s browser diversity.
  • Capture screenshots and videos automatically.
  • Write a great number of tests, including end-to-end tests, integration tests, and even unit tests in JavaScript and tackle asynchronous behavior without test flakiness.
  • Real-time monitoring of test results for immediate insights.
  • Execute tests effortlessly on your local machine or within your CI\CD pipeline.

Сypress, with its App own architecture and built-in browser automation, allows teams to test web applications directly in the browser, eliminating the need for additional drivers or complex setups. Armed with the Gherkin language, Cucumber fosters collaboration between developers and QAs by turning technical steps into plain English narratives. This synergy between these technologies allows teams to carry out comprehensible Cucumber and Cypress end-to-end testing.

What are the benefits of using Cypress and Cucumber?

Let’s discover more closely what these technologies bring into the end-to-end testing and what benefits you can reap with Cucumber and Cypress end-to-end testing:

  • Expressive Tests. Cucumber’s Gherkin syntax allows teams to create human-readable test scenarios and fosters better communication within the team.
  • Powerful Interaction. Cypress provides a rich set of commands for interacting with elements and simplifies the simulation process that includes user actions and interactions.
  • Modularity. Cucumber encourages the creation of reusable step definitions, presents modularity, and reduces redundancy in test code.
  • Real-Time Feedback. Cypress provides real-time feedback during test development as well as allows developers and QAs to preview changes in a jiffy.

Why do teams opt for Cucumber & Cypress test development?

When adopting Cypress and Cucumber for end-to-end testing, teams gain more than just efficient testing tools—they embrace an approach that focuses on clarity and collaboration. Let’s explore the compelling reasons why teams widely opt for these technologies to drive Cucumber and Cypress end-to-end testing:

  • Transparency-based process. BDD with Cucumber unlocks test comprehension for all stakeholders, fostering collaboration within a development team and building trust in the end-to-end testing workflow.
  • Modular and Reusable Test Code. When integrating Cypress with Cucumber, teams can create modular and reusable step definitions to improve the scalability of their test cases.
  • Maintenance. With Cypress’s stable API and robust DOM assertions, teams can reduce flakiness as well as software engineers can concentrate on developing exceptional software.
  • Speed. Cypress’s lightweight design and in-browser execution unleash lightning-fast tests and accelerate feedback loops to keep the development cycle running.
  • Human-Friendly Test Debugging. Cypress’s interactive test runner with time-travel debugging combined with Cucumber’s natural language syntax allows developers and testers to inspect the application state at any point during the test execution. This facilitates easier issue identification and resolution.
  • Embracing the Page Object Model (POM). With POM, teams can create reusable abstractions for UI elements to eliminate repetitive code and maintain concise Cypress tests. This improves code maintainability and promotes a modular structure.
  • Improved reporting. Teams can generate comprehensive test reports and integrate E2E Cypress tests into the CI\CD pipeline to get automated feedback and deploy faster.

Guide: Сreate your first E2E tests with Cypress & Cucumber

Here we are going to dive into the step-by-step guide that will walk you through the process of creating readable end-to-end tests using Cypress and Cucumber with a human-readable behavior-driven development (BDD) approach. You are free to download an example of framework code written in Cypress and Cucumber from our repository of Demo projects.

Step 1: Set Up the Environment for Cypress end-to-end testing

Before starting to write tests, you need to make sure that Cypress and Cucumber are installed. You need to use npm to install Cypress locally and the necessary Cucumber plugins and set up your project structure to accommodate feature files and step definitions.

npm install cypress --save-dev
npm install cypress-cucumber-preprocessor --save-dev

Step 2: Write Your First Feature File

After setting up the environment, the next step is creating an initial feature file using Gherkin syntax. Feature files are used as a narrative for your tests. They help in outlining the functionality that should be tested in a structured manner. Here are some examples:

Feature: User Login

As a registered user
I want to log in to my account
So that I can access personalized content

Scenario: Successful Login
Given I navigate to the login page
When I enter valid credentials
And click the login button
Then I should be redirected to the dashboard

Here, the Feature section introduces the overall feature being tested, and the Scenario section outlines a specific scenario within that feature. The Given, When, and Then steps describe the actions and expected outcomes.

Step 3: Implement Step Definitions

Once your feature test file is in place, the next step involves linking the feature file to the corresponding step definitions written in JavaScript. These step definitions connect the feature file and the actual test code to provide the instructions for each step described in Gherkin syntax. Here is an example:

// cypress/integration/step_definitions/login_steps.js
import {
    Given,
    When,
    Then
} from 'cypress-cucumber-preprocessor/steps';
Given('I navigate to the login page', () => {
    // Cypress command to navigate to the login page
    cy.visit('/login');
});
When('I enter valid credentials', () => {
    // Cypress commands to input valid credentials
    cy.get('#username').type('your_username');
    cy.get('#password').type('your_password');
});
When('click the login button', () => {
    // Cypress command to click the login button
    cy.get('#login-button').click();
});
Then('I should be redirected to the dashboard', () => {
    // Cypress assertion to check redirection to the dashboard
    cy.url().should('include', '/dashboard');
});

These commands interact with your application by simulating user actions. As a result, you can establish a clear link between the human-readable Gherkin syntax and the executable test code.

Step 4: Configure Cypress for Cucumber

At this step, you need to make sure that you have installed the Cucumber preprocessor for Cypress. If not, install it using the following command

npm install cypress-cucumber-preprocessor --save-dev

Then, you should configure Cypress in order to work seamlessly with Cucumber. Just update your ‘cypress.json’ configuration file to include the Cucumber preprocessor:

// cypress.json
{
    "testFiles": "**/*.{feature,features}",
    "supportFile": [
        "cypress/integration/step_definitions/*.js"
    ]
}

If done correctly, this enables the interpretation of Gherkin syntax and linking it to the corresponding step definitions.

Step 5: Run Your Tests in Cypress Test Runner

Once you have configured Cypress and Cucumber, it’s time to execute your end-to-end tests using the Cypress Test Runner. You can run Cypress tests with the command:

npx cypress open

In the Cypress Test Runner, you can see a list of available feature files. From there, you can seamlessly select the desired feature file, and initiate the step-by-step execution of your tests. This interactive process allows you to observe the interactions with your application in real-time and provides valuable insights into test progress and outcomes.

Step 6: View Reports

To improve test result visualization and streamline the bug-fixing process, consider utilizing third-party tools such as testomat.io test management system to view detailed reports. After your test run completes, these tools provide comprehensive information on each test case, including pass/fail status, execution time, and any encountered error messages. By importing your Cypress tests into a Test Case Management System (TCMS), you gain valuable insights into the overall end-to-end testing process and empower your team with detailed reports that facilitate quick issue identification and resolution.

Best Practices with Cucumber and Cypress

Let’s consider the following key practices that will help you write readable end-to-end tests with Cypress and Cucumber:

#1: Apply Descriptive Naming in Test Suite

Use descriptive and consistent naming for feature files and step definitions to improve understanding. If properly adopted, the entire team is in the know about the purpose and functionality of each test component. With clear and intuitive names, you can gain immediate insights into the test’s intent. Furthermore, well-named files and steps contribute to more accessible and maintainable tests, drive collaboration, and reduce misunderstanding within the development team.

#2: Use Gherkin Syntax Effectively

By leveraging Gherkin syntax, teams can articulate software requirements in a clear and human-readable manner. This fosters collaboration and comprehension of test scenarios. Furthermore, this approach allows you to streamline the process of translating business requirements into executable tests that lead to more cohesive and efficient end-to-end Cypress testing.

#3: Organize Feature Files Logically

Logical organization involves grouping feature files based on related functionalities or user stories under descriptive headings. Each team member can quickly locate relevant feature files, contributing to a cohesive understanding of test coverage and encouraging effective collaboration. Furthermore, logically organized files facilitate easy navigation, provide a more scalable and adaptable test suite, and accommodate changes in the application’s features or functionalities with minimal effort.

#4: Craft Modular Step Definitions

Creating reusable step definitions helps you minimize redundancy in your test code and keep it manageable. This modular approach not only improves the maintainability of your test suite but also encourages consistency across test scenarios. Team members can leverage existing step definitions for similar actions to make the end-to-end testing more streamlined and efficient. As your test suite evolves, this modularization allows you to make sure that your end-to-end testing infrastructure remains agile and responsive to evolving project requirements.

#5: Leverage Assertions for Reliable Test Results

Creating clear and meaningful assertions within step definitions ensures that the validation process is transparent and easily understandable. Each assertion should communicate the expected outcome to clearly understand what conditions the tests are checking. This evidence not only speeds up troubleshooting and debugging but also improves collaboration within the team. With Cypress’s assertions, the test code is aligned with the intended behavior of the application leading to a more robust and comprehensible testing suite.

#6: Encourage Team Collaboration

Encouraging a collaborative testing culture ensures that technical and non-technical specialists feel comfortable and empowered to contribute to the effective end testing with Cypress. You need to engage the entire team in discussions about test scenarios, requirements, and improvements to tap into diverse perspectives and insights. This collaborative approach improves the quality of your test suite, promotes shared responsibility, and ultimately leads to a more robust and reliable product launch. Only by establishing open communication channels for discussing test results, feedback, and potential improvements can you foster a collaborative testing culture within the team and all stakeholders, even those without technical backgrounds.

Ready to craft clear and readable Cucumber and Cypress tests?

With the power of Cucumber and Cypress end-to-end testing, you can write E2E tests that are not only reliable and efficient but also clear and engaging for the entire team. These robust end-to-end testing tools offer a seamless integration that improves the efficiency and reliability of your test suite. Armed with the natural language capabilities of Cucumber and the rapid execution features of Cypress, your E2E automated tests become more expressive and maintainable.

Ultimately, with Cucumber and Cypress, you’re well-equipped to navigate the complexities of end-to-end testing and deliver software with confidence.

🎯 Want to invest in readable tests, cultivate a collaborative testing culture, and deliver high-quality software?

👉 Reach out to us for a consultation on how to organize end testing with Cypress and Cucumber. We’ll help you evaluate your needs and existing infrastructure and devise a plan for a smooth implementation of testomat.io test management solution into your existing workflow.

The post How to create readable End-to-End tests with Cypress and Cucumber appeared first on testomat.io.

]]>
Cypress Component Testing and its Implementation (React Example) https://testomat.io/blog/cypress-component-testing-and-its-implementation-react-example/ Tue, 09 Jan 2024 13:45:18 +0000 https://testomat.io/?p=11861 Various testing types are applied to assess the quality of web programs, each playing a crucial role in the software development life cycle. Among them are API testing, End-to-End tests, and component testing. This material will focus on the importance of the latter testing type, specifically in creating quality web applications using Cypress testing framework. […]

The post Cypress Component Testing and its Implementation (React Example) appeared first on testomat.io.

]]>
Various testing types are applied to assess the quality of web programs, each playing a crucial role in the software development life cycle. Among them are API testing, End-to-End tests, and component testing. This material will focus on the importance of the latter testing type, specifically in creating quality web applications using Cypress testing framework.

Why Component Testing?

Before discussing why component tests are crucial in the comprehensive process of testing a web application, it’s essential to understand what this testing type entails 🤔

Component testing assesses the quality of each individual component of a digital product without their mutual integration.

Architecturally, such testing is often equated to unit or module testing, which also involves checking the functionality of individual application parts. However, there are differences between these testing types!

The main distinction between component, unit, module testing lies that in component testing operating on a “black-box” principle, meaning specialized testing engineers know nothing about the internal workings of the digital solution. In contrast, unit testing represents “white-box” testing, where the tester knows how the application functions. This line between Black Box Testing and White Box Testing as difference of Unit and Component testing you can see with the illustration below.

Executing Cypress Component Testing
Cypress Component Testing conduction

The components of a software product can be any of its elements, from the smallest (a button on a website) to complex ones (submitting a registration form). You may use them once or consistently, and they consist of one or several parts. In any case, they are individual program parts that, during component testing, are examined separately without interacting with each other. What components of a digital solution are,

We’ve clarified and their testing relatively to Cypress example:

👉 Imagine you need to verify whether a specific button on a website responds to clicks – that is, how users interact with it. Conducting exhaustive E2E testing or any other comprehensive testing for the entire application would be unjustifiably time-consuming and costly. That’s why no QA team can do without it in their daily practice!

Explore topic, it’s tailored just for you:

Types of Automated Testing Explained Within Test Strategy

In such situations, component testing comes to the rescue. You can quickly check the functionality of a specific component. This is the simplest example of a situation where it is reasonable to test components. So, delve into the advantages of component testing in more detail.

Advantages of Isolating and Testing Components

Every QA team that conducts component testing during the development of a web application gains the following advantages:

  1. Early Defect Detection in SDLC. The quality assurance team can start component testing without waiting to complete the entire application development, thereby identifying defects at the beginning of the workflow. This significantly reduces the cost of bug fixes and saves a substantial amount of project time.
  2. Enhanced Reliability of the Software Product. Since the functionality and performance of each component are assessed throughout the development process, it increases the likelihood of the overall system’s reliable operation.
  3. Reuse of Component Code. Confidence in the high quality of each individual component allows the team to use it again in the development of the digital solution, positively impacting development speed.
  4. Reuse of Test Cases on Project. Additionally, component testing enables the reuse of test environment settings and automation test scenarios when running subsequent component tests. This capability reduces testing time.
  5. Accessibility of Component Testing Regardless of Development Stage. If the QA team needs to check the operation of a specific component, the functionality of which partially depends on other components that haven’t been created yet, this issue is solvable. In component testing, using mocks and drivers to simulate the interface between components is permissible.

A component test can be executed using various testing tools, but in this material, we want to focus on Cypress component testing, providing Cypress component tests for React applications as an example. Both in detail are covered in the official Cypress documentation. By the way, Cypress docs is cool.

Let’s discuss what React components are and the benefits developers and QA team gain by testing them with Cypress. A React component is an autonomous, reusable code fragment that allows dividing the UI into several independent parts, considering them separately.

Benefits of Cypress for React App Component Testing

Cypress supports component testing for React applications out-of-the-box. It is compatible with multiple frameworks, including Next.js, React with Vite, and React with Webpack.

To start component testing in React using Cypress, of course you need available React project. In our case it is a Vite + React project. According to trends, this is the most popular and recommended way to create React app, unlike those build tool that based on outdated Webpack manager. Since nowadays the development policy of React framework relies on the server side. Therefore, this moments should also be taken into account during testing.

Thus, to showcase key concepts of component testing type, we created a basic project. Although you can find in it the essentials like a counter button and multiple links designed. These web elements provide a concise yet comprehensive demonstration of main component testing principles.

Example of Cy test component project with Vite
Simple App to illustrate Cypress component testing
Next follow a few steps:
    1. Install the Cypress testing tool in project folder, for instance using npm:
       npm install cypress --save-dev
    2. Launch Cypress Cloud using the following command:
      npx cypress open
    3. In the opened window, choose the desired testing type; in our case, choose component testing:

      Cy Component testing configuration
      Cypress Component Testing Menu
    4. Configure the project using Cypress Launchpad. During setup, your bundler and framework will be automatically detected.

      Drop dawn to choose a Vite in Cy component testing
      How to pick React Framework Cypress Component testing
    5. Verify the installation of all the required dependencies. It depend from framework. For Vite React build tool with a command:
      npm install -D vite

      If everything is fine, there will be a green checkmark next to each of them. Even if that’s trouble happens, carefully inspect the console. You can find there an instructions on fixing the issue effortlessly. I did it with of the vs of the following commands, learn more about them on LinkedIn post and its discussion.

      npm install --legacy-peer-deps
      npm install --force
    6. Wait for Cypress to generate and add the configuration files to your project.

       Report showcasing the configuration details of Cypress Component Testing
      Cypress Component Testing configuration report
    7. In the next opened window, select the preferred browser:

      Cypress Browser Selection
      Cypress Browser option

After completing these steps, you can begin writing your first test and running actual component tests in your spec file.

A basic example of a Cypress component tests for React App may look like:

import App from '../App.jsx'

describe('ComponentName.cy.jsx', () => {
  it('mount', () => {
    cy.mount(<App/>)
    cy.get('button').click()
    cy.contains('button', 'count is')
  })

  it('Test a Message', () => {
    cy.mount(<App/>)
    cy.get('p.read-the-docs').should("have.text",'Click on the Vite and React logos to learn more')
    })

   it('Logo', () => {
      cy.mount(<App/>)
      cy.get('img.logo.react').click()
      cy.wait(2000);
      })

      it('Header', () => {
        cy.mount(<App/>)
        cy.get('img.logo.react').click()
        cy.wait(2000);
        cy.url().should('include', 'https://react.dev/');
        cy.contains('main h1', 'React').should('be.visible');
 cy.get('div.hidden.sm:block').should("have.text",'Help Provide Humanitarian Aid to Ukraine')
        })

})

Understanding the Cypress Folder/File Structures:

Structure of Cypress component testing project

  1. Components folder: Contains Cypress component specs
  2. Fixtures folder: Any data which can be used inside the specs can be placed here
  3. Support folder: helper functions, utilities, any reusable code, and custom commands can be placed here.
  4. Support/component.js: Great place to put the global configuration and reusable code
  5. Support/commands.js: In this file, you can create custom commands or override existing commands
  6. Cypress.config.js: This file contains runtime configurations such as devServer, framework, bundler, reporter, videos, screenshot options, etc.
  7. Package.json: This file tracks all installed dependencies and allows you to create a custom commands shortcut

🎉 Hurray, the result we got!

 Execution of Component Tests using Cypress Cloud
Running Component tests with Cypress Cloud

🛠 Handy CLI commands of Cypress Component Testing

npx cypress run --component   //Execute component Tests in headless mode.
npx cypress run --component --config video=false   //use the command to to define the Cypress configuration.
npx cypress run --component --headed   //Execute Cypress component tests, directly in the headed mode without manually selecting the test files, browser and tests on the test runner window. 
npx cypress open --component --browser chrome   //Open the test runner window directly by skipping the browser option. 
npx cypress run --component --spec cypress/component/ComponentName.cy.jsx

Follow this link to know how to Run Cypress Component tests alternatively:

The Ultimate Cypress Tutorial: How to Organize your Advanced Testing Framework

Key Features of Cypress Component Testing

If you’re in the process of selecting a test framework for conducting component testing for your application, you should consider Cypress for the following unique features in the QA process:

  • Snapshot Capabilities. Cypress takes snapshots during test runs, allowing you to hover over Cypress commands and instantly see what happened at that point in testing.
  • Fast, Efficient Debugging of Tests. Understanding why specific tests fail is straightforward in Cypress. The stack traces available in the framework and familiar tools like Developer Tools make debugging tests simple and high-quality.
  • Automatic Waiting. The tool defaults to executing a command only when the page elements are ready. There’s no need to add sleep or wait statements to your component tests.
  • Stubs, Spies, and Clocks. Cypress provides commands like cy.stub(), cy.spy(), and cy.clock() to control the behavior of functions and time intervals in your application.
  • Network Traffic Stubbing. You can test edge cases without involving your server.
  • Reliable Testing Results. Cypress’s architecture doesn’t rely on Selenium WebDriver, ensuring you get reliable and fast component tests.
  • Visualization of Test Results. The framework allows you to view screenshots and video recordings of tests, making it easy to identify the cause of failures and resolve them more quickly.
  • Unreliable Test Detection. Cypress has a built-in Flaky Test Management System that eliminates tests showing incorrect results.

All these tool capabilities make Cypress component testing fast and reliable, enabling teams to release high-quality software products in a short timeframe.

Advanced Cypress Component Testing Techniques

Cypress is a versatile web application testing tool that supports various tests, progressive testing approaches, including BDD approach and parallel test execution using cloud services. Let’s delve into advanced techniques that make component testing with Cypress even more effective.

Cross-browser Testing

You can run component tests in multiple browsers: Chrome, Electron, Microsoft Edge, Safari, and Firefox. You can find the full list of available browsers in the Cypress menu.

Specify the browser for testing using the –browser flag or a specific command. For instance, the command would look like this for Firefox:

cypress run --browser firefox

For added convenience, use scripts:

Note: To run tests on a CI pipeline, its containers must be installed locally or in the cloud. We’ll further discuss running Cypress in continuous integration mode.

Parallel Testing

Cypress supports parallel test execution without additional requirements. To parallelize a test suite, simply use the following command when running tests in CI:

cypress run --record --parallel

Intelligent Orchestration

After setting up recording in Cypress Cloud, additional capabilities become available. For instance, you can rerun failed specs using Spec Prioritization and cancel test runs with Auto Cancellation in case of unexpected failures.

Recording Tests in Cypress Cloud

Cypress users can access the Cypress Cloud web component, providing detailed testing results. Here, you can view videos and screenshots, check the ratio of skipped, passed, failed, and pending tests, inspect the stack trace, and more.

Summarizing

Component tests are a crucial component of the software development life cycle. A quality assurance team can ensure that each individual component of a digital solution works as expected. This, in turn, contributes to the creation of higher-quality and more reliable products that working properly.

Such tests can be executed in various frameworks, also an actively utilized tool for web application component testing is Cypress. It supports parallel and cross-browser testing, allowing tests to be run locally and in CI mode.

If you have any questions regarding component testing or any other types of testing, including their automation, feel free to reach out to the testomat.io team. We are ready to address each inquiry and make your QA processes more effective and streamlined.

The post Cypress Component Testing and its Implementation (React Example) appeared first on testomat.io.

]]>
🚀 Cypress and REST API Testing https://testomat.io/blog/cypress-and-rest-api-testing/ Thu, 04 Jan 2024 21:10:29 +0000 https://testomat.io/?p=11989 With API testing, teams verify the proper functioning of backend and front-end services, detect potential issues early in the development cycle and minimize overall project risks. Many teams leverage Cypress for API testing. Cypress API tests allows modern development teams to comprehensively validate both front-end and back-end functionalities within a single framework and drive a […]

The post 🚀 Cypress and REST API Testing appeared first on testomat.io.

]]>
With API testing, teams verify the proper functioning of backend and front-end services, detect potential issues early in the development cycle and minimize overall project risks. Many teams leverage Cypress for API testing. Cypress API tests allows modern development teams to comprehensively validate both front-end and back-end functionalities within a single framework and drive a cohesive testing strategy in their workflow.

😀 So let’s find out more about Cypress framework and REST API testing and what benefits it can bring into your workflow.

What is API testing?

API testing is utilized to confirm the functionality, performance, and security of application programming interfaces (APIs). Generally, APIs help provide communication and data exchange between diverse applications and services as well as allow them to interact and share information.

With testing the APIs, you can test various aspects of an API, including its endpoints, input/output data, error handling and so on to make sure that the API behaves as expected and meets the requirements of the apps/services relying on it.

What are the categories of API testing?

  • Web APIs: With HTTP or HTTPS protocols, they are used for data exchange in a client-server architecture.
  • REST APIs: Using HTTP methods like GET, POST, DELETE and PUT to perform operations on resources, they follow a set of architectural principles.
  • GraphQL APIs: this type allows clients to request and get specific data they need and offers an efficient approach to data fetching.
  • SOAP APIs: they use XML-based messages over protocols like HTTP, SMTP, or others for communication.

Why do teams choose Cypress for REST API testing?

Originally created to address the challenges and limitations of traditional end-to-end testing frameworks, Cypress Javascript frontend testing framework provides a developer-friendly approach, robust features, and seamless integration into the development workflow. Operating in the browser, it runs directly alongside your application and has more control over the browser and the ability to interact closely with the application.

Development and testing teams can easily install and set up it as well as simply add it as a dependency in the project or use it as a standalone tool. In addition to that, Cypress comes with a user-friendly interactive Test Runner that displays live test results, logs, and interactive debugging tools.

👉 Let’s go more in-depth into some of the reasons why teams opts for Cypress when conducting REST API testing:

  • With Cypress, teams can seamlessly integrate both end-to-end (E2E), Component and testing the API within a single platform. This helps them maintain a consistent test automation strategy across different layers of their applications.
  • Thanks to JavaScript-based syntax, teams can leverage existing JavaScript skills for both frontend (E2E) and backend testing as well as drive efficiency and code reusability.
  • In terms of in-browser testing capabilities, teams can visually observe API interactions in real time, get immediate feedback and improve the debugging process.
  • Cypress’s automatic waiting mechanism eliminates the need for explicit waits in API tests. Before executing commands, it waits for the app to be stabilized. This reduces flakiness and improves the tests reliability.
  • With Cypress’s time-travel debugging feature, developers can inspect the state of API calls at different points during test execution. This ultimately makes the issues identification and resolution easier.
  • With easy setup and configuration, teams can quickly adopt Cypress into their testing workflow without difficulties.
  • With a rich set of comprehensive commands specifically designed for API testing, teams can cover a wide range of functionalities, from making HTTP requests to handling authentication, simplifying the process of creating effective API test scripts.
  • Thanks to real-time reloads and dual-view features, teams can dynamically reload the API configuration or specifications during development as well as display simultaneous viewing of test execution and API interactions.
  • Teams can combine API tests with frontend tests to cover end-to-end scenarios and make sure that interactions between the UI (user interface) and the API (application programming interface) are smooth.

To sum up, teams choose Cypress in API testing in terms of its seamless integration and the ability to simplify testing strategies across different layers of apps. Cypress’s JavaScript-based syntax, real-time in-browser testing, and unique features, such as time-travel debugging, offer a developer-friendly and efficient approach for testing, streamlining it.

What are the benefits of Cypress REST API Testing?

Here we are going to overview what key advantages testing with Cypress brings in the testing workflow:

  • Real-Time Reloads. With Cypress features, you can automatically reload the application in real time and make appropriate changes to your test code if there is a need. This makes the development and debugging process highly efficient.
  • Fast Execution. With Cypress fast test execution, you can minimize the common flakiness issues in end-to-end testing and provide reliable and quick feedback to developers.
  • Debugging Capabilities. With Cypress, you can pause and inspect the state of your application at any point during the testing process and simplify the bug fixing process.
  • Simplified Syntax. When it comes to writing tests, Cypress requires less boilerplate code compared to other testing frameworks. This speeds up the learning curve of developers with varying levels of testing experience.
  • Built-in Assertions and Commands. With Cypress built-in assertions and commands features, you can make the process of writing and organizing tests easier.
  • Parallel Execution. Thanks to Cypress, you can run tests in parallel. This speeds up the software testing process and improves efficiency, especially when running tests in a continuous integration or delivery pipeline.
  • Test Data and Mocking. With Cypress, teams can manage test data, mock API responses and validate status code. It provides using fixtures to load test data from external files and creating custom Cypress commands to encapsulate API-related logic.

What are the challenges QA teams face in Cypress REST API testing?

Here we are going to overview some challenges the teams face when conducting Cypress API testing:

  • Limited HTTP Method Support. In terms of focusing on e2e testing tailored for interacting with the UI, Cypress’s support for various HTTP methods and edge cases in API testing might be more limited compared to other API testing tools.
  • Lack of Granular Request Control. When conducting API testing, you need precise control over HTTP requests, including headers, query parameters, etc. Primarily designed for UI interactions, Cypress might not provide the same level of granularity and control.
  • Data-Driven Testing Challenges. Compared to other tools specifically designed for testing, Cypress may not offer built-in features for efficient data-driven testing to handle a variety of data inputs and scenarios.
  • Dependency on UI for Setup. As Cypress has a focus on interacting with the UI, there are additional complexities when trying to set up test data or authentication without involving the UI components.
  • Test Execution Speed. Known for its fast test execution in the context of end-to-end testing, API testing might require repetitive and rapid execution of numerous requests. The overhead of the Cypress architecture may impact the speed of execution for API tests.
  • Learning Curve for Testers with API Focus. QA engineers may find Cypress less intuitive for API-specific scenarios. The learning curve might be steeper for those who are more familiar with tools specifically designed for testing the API.
  • Limited Reporting and Documentation for API Tests. Cypress’s reporting and documentation features are more tailored towards end-to-end testing scenarios. Reporting for API tests might not be as comprehensive or customizable compared to dedicated tools for testing APIs.

How to perform Cypress API testing

Before diving into automated Cypress API testing, let’s overview some of the key commands:

  • cy.wait() facilitates the loading of async data and allows waiting for a network request.
  • cy.route() directs test requests to specific handlers.
  • cy.server() helps to maintain an entire server for a test suite.
  • Test Runners: provides parallel execution of tests for quick turnaround time.
  • cy.intercept() keeps the responses under control or simulates the behavior by intercepting the requests.
  • cy.request() allows you to send HTTP requests and interact with APIs directly within your test cases.

Prerequisites

  1. You need to install or update Node.js to make the HTTP requests to the API server.
  2. You need to check if you have a version of Node.js compatible with Cypress. *On the moment of writing this article Cypress supports minimum Node.js 14+ version.
    node -v
  3. You need to install your chosen code editor and configure it according to your preferences.

Once all the prerequisites are in place, you can follow these steps to carry out automated Cypress API testing:

Step 1: Set Up Your Cypress Project

Start from creating a directory for your Cypress API testing project named cypress-api by navigating to your preferred location with the following commands

mkdir your_cypress_api
cd your_cypress_api

By running the mentioned above command, you need to initialize a Cypress as a project dependency to automate web interactions and perform Cypress API testing:

npm install cypress --save-dev 
npx cypress open

Step 2: Write Your First API Test

When writing a simple Cypress API test script, you can use the cy.request() command to initiate HTTP requests, specifying the method, URL, headers, etc. This command allows you to perform testing with ease and provides assertions on the response status, body, headers, and other relevant details. Here is an basic example that demonstrates the simplest API response.

In this example, test hits the API server with the URL and checks whether a valid login request returns a status code of 200. This means the status code to be 200 for successful test execution.

describe("HTTP Requests", ()=>{
    it("Get Call", ()=> {
        cy.request({
            method:'GET', 
            url: 'https://reqres.in/api/users/2',
        })
        .then(function(response){
            expect(response.status).to.equal(200);
            cy.log(JSON.stringify(response.body));
            console.log(response.body);
        });
    })
});

Meet 👇

Simple examples of REST APIs tests for starters using Cypress from official site

**Examples of uses of the Cypress API from Cypress Doc

Step 3: Run Cypress Tests via Cloud

Once you’ve written your first Cypress API test or already had the tests, the next step is to execute them. Running your tests allows you to validate the functionality and reliability of your API endpoints. Use npx cypress open command to open the Cypress Test Runner with an available list of your test files for your project to be executed.

Execution API tests using Cypress Cloud
Run API test with Cypress Cloud

You can choose to run all tests in the file or select specific ones for execution and ensure thorough examination of status code or status codes during the testing process. As the tests run, Cypress provides real-time feedback in the Test Runner interface. This helps you observe each test step, including API requests and assertions, in the interactive dashboard.

Step 4: Analyze Test Results

After test execution is complete, Cypress displays a summary of the results. You can discover information not only about passed and failed tests but also about errors. This helps you inspect test details, logs, and snapshots of your application. Based on test results, teams can make improvements and address issues.

Step 5: Integrate API Tests with CI\CD

Consider integrating your Cypress API tests into your Continuous Integration/Continuous Deployment (CI\CD) pipeline for automated testing. This helps you provide consistent validation with each code change and contributes to a continuous testing workflow. This practice also improves the overall quality and reliability of your software by validating the API endpoints throughout the development process.

Step 6: Visualize Test Results

Opt for third-party tools like testomat.io test management system helps you visualize test results.

Cypress test cases imported into Testomat.io test management
Cypress tests imported into TCMS
Cypress test report of run execution
Visual Run report for Cypress tests
Simple Visual representation of Cypress test results in execution
Simple Cypress test Results’ execution

With clear visual feedback, you can quickly identify successful builds and pinpoint any issues that require attention. In addition to that, continuously iterate your Cypress API test scripts based on feedback and changes in your API. This allows you to improve your test suite and add more tests to handle different API scenarios.

More detail step-by-step:

Best Practices for Writing Robust API Tests in Cypress

These best practices help you maintain the highest quality of your test data and also make sure that each team member keeps up with relevant changes in the software testing process.

#1: Think of Modularization and Reusability

When it comes to creating Cypress API tests, you should organize your API tests into separate modules based on functionalities or endpoints to improve test structure. You can also apply Cypress custom commands to encapsulate complex API interactions that promote a cleaner and more readable test code. With modularization and reusability, you can adapt to changes in the API, facilitate easy updates, minimize repetition, and drive a scalable and efficient testing process.

#2: Establish a Structured Approach to Test Organization

Organizing tests into logical structures using describe blocks based on the functionality or feature they are testing is essential. Also, don’t forget about maintaining consistent names for test files and describe blocks. This helps team members quickly understand the content. A well-structured test organization not only improves readability but also simplifies test maintenance and collaboration within the development team.

#3: Encourage Continuous Debugging

A culture of continuous review and refinement of your tests helps you make sure that tests adhere to coding standards. Fast identification and resolution of flaky tests are vital. With automated monitoring tools, you can detect flaky tests, report inconsistencies in test behavior and resolve them. When encouraging regularly scheduled code reviews coupled with refactoring, teams drive a collaborative environment where code quality is continually improved and the codebase remains maintainable.

#4: Keep documentation up to date

A well-maintained API documentation becomes an invaluable resource for onboarding new team members and supporting collaborative efforts to maintain a robust API test suite. This also drives a shared understanding within the development team. Regular updating the API documentation allows each team member to better understand expected responses, error codes, and data structures and resolve any issue as quickly as possible.

#5: Incorporate Test Data Management System

Keeping test data closely associated with the tests provides clarity and simplifies maintenance. With a well-organized and efficient test data management system, you can regularly review and update test data to align with evolving application requirements. This improves visibility and comprehension, speeds up easier updates and troubleshooting when necessary and guarantees accurate testing outcomes.

Ready to drive innovation and use Cypress for API testing?

Cypress appeared to be a dynamic, powerful and user friendly testing framework to perform API testing. It not only establishes a developer-friendly ecosystem for creating resilient API tests but also provides unique functionality to test the integration between the UI and the API.

With Cypress, you invest in an efficient, reliable and scalable software testing process that helps you deliver the software products of high quality.

👉 Drop us a line if you want to delve deeper into the realm of REST API testing with Cypress and discover more testing practices!

The post 🚀 Cypress and REST API Testing appeared first on testomat.io.

]]>
Cypress Dashboard, Cypress Cloud its Advantages and Disadvantages https://testomat.io/blog/cypress-dashboard-cypress-cloud-its-advantages-and-disadvantages/ Wed, 27 Dec 2023 15:43:13 +0000 https://testomat.io/?p=11923 To ensure that software testing yields the expected results, it is not enough to simply create test cases and run them in the chosen testing environment. Another equally important aspect is obtaining high-quality reporting on test results. Most modern testing automation platforms provide effective tools for creating reports, allowing tracking of skipped, passed, and failed […]

The post Cypress Dashboard, Cypress Cloud its Advantages and Disadvantages appeared first on testomat.io.

]]>
To ensure that software testing yields the expected results, it is not enough to simply create test cases and run them in the chosen testing environment. Another equally important aspect is obtaining high-quality reporting on test results. Most modern testing automation platforms provide effective tools for creating reports, allowing tracking of skipped, passed, and failed tests on a project and, more importantly, understanding the reasons for errors.

In Cypress, all the mentioned capabilities and many other features are implemented in Cypress Dashboard, which its creators recently renamed to Cypress Cloud. We will delve into what it is and how to work with it ⚙

What is Cypress Dashboard

Using the Cypress test runner, you can run tests in parallel with your app, allowing you to see assertions, network requests, browser events, and other commands directly in the browser. This contributes to the quick execution and debugging of tests.

When running tests in a CI pipeline, visibility into the process is partially lost. CI essentially represents a black box that does not allow you to fully see how your app behaves. This is a significant problem for testing modern web apps, as they are highly dynamic and require a high level of visibility and sufficient details for debugging.

Cypress Dashboard helps address these and many other issues – a web component that provides full visibility into your test runs in the CI pipeline, allowing you to control their status and examine reports on test results. Additionally, it speeds up testing, accelerating the release of high-quality software products.

To see all the necessary information for a Cypress project, click on its name on the dashboard. Among other data, you will have access to information about the number of passed and failed tests and the duration of the test run.

You can filter and group all test runs by various properties – this feature is especially relevant when working on large projects where developers add large amounts of code daily. Among other dashboard features:

  • Viewing statistics for skipped, passed, failed, and pending tests.
  • Examining the stack trace of failed test cases.
  • Representing spec files as a histogram allows you to visually identify those that took the maximum amount of time to execute.
  • Support for screenshots and videos for even greater clarity when examining testing results. Note: you can view both the full video of the entire test run and its excerpt.
  • Test parallelization feature speeds up the testing process as the codebase and test run duration increase. All that is required is to specify cypress run – record – parallel when launching it in CI. After that, Cypress will automatically determine how to balance the load of all spec files on multiple machines in the CI.

As such, Cypress Dashboard is an indispensable tool for all users of this framework who run testing in the CI pipeline and want to achieve optimal visibility throughout the testing process from start to finish.

How to Run your testing with Cypress Dashboard

To create and run a Cypress test, you first need to install the framework. You can do this in several ways:

Via npm:
npm install cypress --save-dev

When choosing this option, it’s important to ensure that you have npm init running, a node_modules folder, or a package.json file. This ensures that Cypress is installed in the correct directory.

Via Yarn:
yarn add cypress --dev
Via pnpm:
pnpm add cypress -D

The all installation details you can find in the official Cypress Docs that definitely is a strong point of this testing framework.

After installing and opening the framework, you can open the Cypress App with the command:

npx cypress open

After choosing of E2E or Component Cypress types of tests Create new empty spec file with the button to create your first test:

Initiating a New Testing Scenario with Cypress Cloud
Creating a new Cypress spec file

Now, add a name for the new specification and create your first test file within it; let’s call it first-test.js.

Next, run tests in the Cypress window or through the Command Line Interface (CLI) using the commands

npx cypress run

Note one more time, that Cypress Dashboard recently changed its name to Cypress Cloud, to enable the test recording in your Cypress Cloud you should follow the Dashboard tips and establish run connection to a project within the interface and check the project settings cypress.config.js file.

Configuring Cypress Cloud Settings for Test Imports
Cypress Cloud settings to import tests

After then run the following command:

npx cypress run --record --key <your key>
History of Runs Cypress Dashboard

As a result follow through the link of run and you can track result of your tests in Cypress Cloud as well.

Test Run Results Cypress Cloud

Cypress Tests on CI\CD from Cypress Cloud

Another capability the Cypress framework provides is the ability to run tests on CI/CD. Smart Orchestration of Cypress Cloud enables you to run tests with your favorite CI\CD tools in your CI environment and coordinates runners and balances test loads out-of-the-box. Essentially, to run Cypress tests on CI/CD, as example GitHub Action only 2 steps are required:

Setup Cypress tests on CI\CD

Of course, that depending on your CI provider, you may need a specific configuration file. Information about the required file and its installation can be found in such tips. After setting up testing in the CI pipeline, Cypress will record tests and save test results in the Cypress Dashboard.

Test execution results includes test run statistics, specific failure details, and deep links to results in Cypress Cloud for fast debugging.

Cypress Cloud’s Spec Prioritization lets you prioritize recently failed specs so they run faster on their next run. The Auto Cancellation feature will cancel a whole test run on failure, saving time and resource usage. You can also cancel in-progress runs manually from Cypress Cloud if you need to.

Cypress Limitations 😢

Some limitations in the Cypress framework lead QA teams to seek alternatives to this platform. These include:

  1. The framework does not support working with mobile applications. The tester can only check specific functionality in the mobile browser.
  2. Cypress works only with one programming language – JavaScript.
  3. Simultaneous control of multiple browsers and support for multiple tabs are not available.
  4. Test results tracking is possible through Mocha.js reports, on which Cypress is built, or by using the web component Cypress Dashboard.
Cypress Pricing

On the other hand, there are also open-source alternatives, which we would like to discuss with you at the bottom.

A free Cypress Dashboard Alternative

Cypress Dashboard is the most popular solution for tracking results of Cypress tests, used by global giants like PayPal, Disney, etc. However, this tool comes with a relatively high cost, which can be a significant issue for small teams using this framework.

Fortunately for Cypress users, Cypress Cloud has a free alternative – Sorry Cypress and its enhanced version Current. They are used for parallelizing, recording, and debugging Cypress tests. They are open-source and has an MIT license.

It is worth noting that debates are ongoing in the testing community regarding the legitimacy of having a free alternative to Cypress Dashboard. However, the projects are not closed at present, and all framework users can work with the alternative panel starting from version 6.7.0.

Key features of Sorry Cypress project :

  • Parallel test execution for faster testing in the CI pipeline.
  • Group tests by browser or device in a single test run.
  • Identify flaky tests.
  • Record videos and screenshots and access the stack trace.
  • Track the duration, status, and progress of test run, spec files, and individual tests.
  • Integration with GitHub, Slack, MS Teams, Webhooks, and other third-party services.

As you can see, Sorry Cypress’s functionality is on par with the original solution – Cypress Cloud. The only question remaining is the ethics of using free software as an alternative to the official digital solution from the Cypress framework developer. And on a moment of article publication it works with elder Cypress versions.

Testomat Differences with the Sorry Cypress

Test management system testomat.io is a modern test management system for working with Cypress tests. This is made possible by the convenient TMS function – importing automated tests from other systems. After importing Cypress tests, users can run tests and use Project Dashboards in test management to study test results and their deep analytics.

Project Dashboards in Testomat.io offer capabilities different from Sorry Cypress. Here’s what we provide:

  1. Tests: Users can access all actions with tests in this dashboard. In addition to viewing test cases, users can create, delete, or import tests.
  2. Runs: All test runs are stored here, and you can view results at any time. Users have access to graphs based on test status – failed, passed, or skipped tests.
  3. Parallel test execution: this testing tool supports both sequential and parallel QA processes. Unlike Cypress, TMS supports testing web applications and mobile digital solutions.
  4. Also available here are the sections Runs Archive and Groups Archive: They contain information about tests that were run previously. You will be able to see the date of their execution, the time spent, the ratio of tests based on their status, as well as their total quantity in each test run.
  5. Pulse: This dashboard allows users to recover tests deleted within the last 90 days. Users can also analyze all changes made to them, filtering content based on various indicators, such as the date of change or the responsible user.
  6. Imports: This dashboard contains information about when and which tests were imported into TMS. Users can track the dynamics of testing here.
  7. Analytics: Widgets in this dashboard provide QA teams with deep project analytics. Information includes metrics such as:
    → Automation Coverage – the number of automated tests.
    → Failures – defects in the project.
    → Ever-Failing Tests – tests that have not received necessary fixes for a long time.
    → Slowest Tests – tests indicating performance issues or automation errors.
    → Flaky Tests – tests showing different results in each test run despite no changes in the test code.
    → Never Run Tests – test cases imported but never run.
    → Tags Statistics – automation percentage and test status based on used tags.
    → Environments – the same metrics as the previous point, but broken down by test environments.
    → Jira – information about tests linked to the Jira system.
  8. Wide integration possibilities: our test management solution seamlessly integrates with the Jira system and popular CI providers, including GitHub, GitLab, Jenkins, etc.
  9. Team collaboration on a project: This is primarily possible thanks to the clear Living Docs documentation and BDD-format support of tests.
  10. Plans, Steps, Branches, Settings, Help. While not directly related to tracking test results, these dashboards, available in Testomat.io Project Dashboards, offer additional functionalities:
    → Users can create, modify, delete, or launch individual test plans in the Plans menu.
    → Steps represent a database of steps that can be reused when creating test cases, speeding up the process.
    → In Branches, users can create, modify, and delete test set branches, allowing testers to work on different tasks simultaneously, speeding up the testing process.
    → Converting manual tests to automated. Simply write test scenario steps and code for their automation – TCMS will mark the necessary tests as “automated.”

The Settings menu provides TMS users with many options for optimizing testing. Regarding reporting, users can modify test result message templates and configure notifications for completed test runs.

Sum Up: Is Cypress.io Dying?

Cypress was initially positioned as an open-source solution with rich functionality, quickly gaining a large audience of enthusiasts. However, over time, many users were disappointed by the introduction of paid web components necessary for the framework’s operation – one of which we discussed today, namely Cypress Dashboard.

This situation prompts many Cypress users to seek alternatives to the familiar tool, opting for Playwright, Selenium, or other platforms with similar functionalities. Despite this, it’s too early to speak of the definitive “demise” of Cypress.io – teams continue to work on it, particularly those finding it challenging to transition their projects to other platforms or those guided by personal preferences.

The post Cypress Dashboard, Cypress Cloud its Advantages and Disadvantages appeared first on testomat.io.

]]>