Clara

QA Document

Title of RFC

Introduction

This document serves as an RFC (Request for Comments) for the QA strategy and approach to be implemented with the QA project(Clara).

Objectives

The primary objectives of Clara QA project is to use the capabilities of Playwright in Automating UI(Clara Admin Dashboard) and API testing to ensure the product quality, reliability and verify API standards within defined timelines. Clara QA project will not focus on Unit Testing, Performance Testing and Security testing for now. It also doesn't specifically address test data management such as test data generation and data cleansing or data manipulation. We will use the client specific data(Public website URL) to perform functional & API tests.

Motivation

Testing plays a critical role in the success of any software application. The primary usecase of Clara is to integrate knowledgebases and provide search solution using a command bar. By automating this clara application with functional test cases, we aim to achieve the following: Delivering high-quality software, mitigating risks with compatibility testing, product's reliability through comprehensive testing.

Design Proposal

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

Official Documentation Reference - Playwright Documentation using Node.js (opens in a new tab)

Testing Framework

The proposed testing framework comprises the following components:

Playwright: Utilize Playwright's cross-browser/cross-platform automation capabilities to interact with web applications, simulate user actions, and validate expected behaviors.

Node.js: Leverage the Node.js runtime environment to execute Playwright scripts and harness the benefits of asynchronous programming.

Reporting: Integrate a reporting mechanism, such as HTML reports, to provide detailed insights into test execution results, including pass/fail status, screenshots, and error messages.

Testing Approach

The testing process will follow these steps:

Test Case Design: Create comprehensive test cases covering functional and API testing scenarios. Define test steps, expected outcomes, and any necessary preconditions or test data.(We will start working on Test Plan doc once wireframes are ready)

Test Data Preparation: Develop a strategy to manage test data. Utilize static test data files for common scenarios and dynamic data generation techniques for varying test inputs.

Test Script Development: Implement test scripts using Playwright's APIs to interact with web elements, perform actions, and validate results. Leverage the powerful features of Playwright, such as page objects and selectors, to enhance test script maintainability and reusability.

Test Execution: Execute the test scripts using the selected test runner, leveraging parallel execution capabilities to optimize test execution time.

Reporting and Analysis: Generate detailed test reports that provide insights into test execution results, including individual test case outcomes, logs, screenshots, and error details. Analyze the reports to identify failures, trends, and areas for improvement.

Drawbacks

Playwright can not be used for performance and security testing which is out of scope for now.

Alternatives

There are some paid alternatives to Playwright such as testim, rainforest, Katalon studio etc. However, considering the QA requirements, budget, and team's skillset: Playwright meet the needs of our Clara QA project without requiring any significant upfront investment.

Questions and Discussion Topics

Based on Clara Architecture discusssion to ensure a streamlined and controlled release process, testing workflow happens from development to production as follows:

Feature Branch -> PR for code Review and QA -> Deploy manually to Staging Branch -> End to End QA review on Staging Environment -> Tag the release with a semantic version -> Release Manually to Production -> Basic QA on Prod & QA Monitoring