Tag: #Vitaq

Why Vitaq AI and why now?

Vitaq is equipped with AI machine learning algorithms that are coverage led. When you run Vitaq in AI mode, it will use the coverage model (defined by you the user) as the goal to try to achieve 100% coverage. It will do this by working out what is the most optimal set of tests to achieve this target.

The variable nature of your Vitaq Test Activity will provide many next allowable action selections and many different test data inputs. By informing the Vitaq AI machine learning algorithm what you care about, i.e. what user journeys you want to test and what test data you want to be generated, Vitaq will continuously run until it achieves 100% coverage of your coverage model.

The reasons to adopt Vitaq AI Test Automation:-

1) It helps you extend your testing from your traditional test Mocha (or other) Frameworks by creating a Vitaq AI Test Activity diagram by re-using all of your test framework JS code.

2) Your new Vitaq Test Activity allows the user to apply AI Machine Learning algorithms to auto-generate tests that focus on the things that you the QA Tester really cares about, which is your user journeys.

3) Vitaq AI gives users the ability to continuously generate tests which are highly variable, data-driven and which will explore the test space in ways which QA Testers may not necessarily think about.

Advantages of using Vitaq AI Test Automation:-

Vitaq AI delivers the following benefits:-

1) Productivity.

2) Efficiency and Effectiveness.

3) Reducing Risks.

Let’s discuss these three characteristics in more details :-

1) How Vitaq AI helps to improving productivity :-

It helps to improve productivity because you are able to capture your complete test space with a graphical visual model which is our Vitaq AI Test Activity.

This graphical test model, helps to drive the business logic of testing out into the open for all of the project contributors to see. It helps to improve communication between your managers, developers, dev testers and QA testers.

By releasing the power of AI machine learning to “explore” your newly created Test Activity Diagram (model of your test space) Vitaq AI will use compute power to auto-generate different test scenarios of test action script after test action script. All of these test action scripts can be be enhanced with highly variable auto-generated test data to give you data-driven tests. Because Vitaq AI by its very nature will give you highly variable tests, it does things which test developers might not think about, which helps it find defects which other approaches miss.

2) How Vitaq AI helps to improving efficiency and effectiveness:-

Vitaq AI Test Automation Tool helps in utilizing the test scripts that you may have already written for Mocha or other test frameworks, then enables you to make them data-driven and reusable across the test space.

With Vitaq AI the user is able to complete their testing by leading with QA Coverage. The user journey coverage capability is a unique way of not only monitoring, measuring and analyzing your user journey test coverage
but becomes the very goals that the AI machine learning algorithms of Vitaq will use to auto-generate your tests.

3) How Vitaq AI Helps in Reducing Risks :-

Vitaq AI uses the power of variability which auto-generate tests that quite often are not created in conventional frameworks like Mocha. So it will auto-generate tests that QA testers may well have missed, This helps find defects the other approaches miss and knowing that you have covered all of the user-journeys you care about, helps in reducing risks.

As well as completing all of your test requirements by achieving 100% QA Coverage, Vitaq AI can be run continuously to ‘explore’ in your test space to find corner cases that may escape into production. You can leave Vitaq AI running continuously utilising your spare compute resource 24 hours a day, 7 days a week. This will reduce your risks by helping to find more bugs earlier and make sure they get fixed before delivering your software build to clients or stake holders.

Vitaq AI can help take your Test Automation to the next level, so if you want to release the power of AI machine-learning combined with your QA testing skills, then the Vitaq AI-driven, coverage-led, auto-generated tests will make your testing life easier.

About AutomationPractice.com

We will be using the AutomationPractice.com e-commerce web application site to walk you through how to apply AI-driven continuous test automation with Selenium and WebDriverIO.

Automation Practice is an Ecommerce Website to help you practice test automation exercises on a real-time e-commerce website.

It consists of different workflows for adding products to cart, checking out
from the cart, making payments, etc.

Screenshot for adding products to cart​ :-

Screenshot for checking out from the cart​ :-

Screenshot for making payments​ :-

Workflows can be automated using Test Automation scripts and this website is
a great practice candidate for doing that​ .

To help you assess what our new Vitaq AI Test Automation Tool can do to help you get more testing done in less time, we have started our blog with a description of the classic way that many many QA and Dev testers deploy their testing using Jira for planning and then a conventional test framework such as Mocha. Once we have quickly stepped you through what you know already, you can easily relate it to how we tackle the test project with the new Vitaq AI continuous test automation tool, helping you to answer the key questions of Why Vitaq AI? and Why Now?

Gathering Test Requirements into Atlassian Jira

Before we jump into showing all the the new new AI-driven continuous test automation things in Vitaq we first need to talk about test planning. Failing to plan is planning to fail, so let’s begin with success in mind and take a look at how we approach test requirement planning. There are many ways to gather our web or mobile applications test requirements into a test plan. We typically take our story board and break it down into user stories and then into detailed user-journey definitions. One of the most common ways to capture this is to document them directly into Jira. Let’s look at how we did this for this example project :-

1) Start with our Product Requirements which needs to be tested.

For Example:- As mentioned in the earlier Blog we have chosen to use the AutomationPractice.com application site for our example project because it is representative of many e-commerce websites.

For this example project we have identified many test requirements which include searching for a product, adding it to cart and completing the payment process.

2) Communication of key features.

Our Test Cases need to communicate the key features of the application under test.

In our example project, one of the key features is to purchase items successfully from the registration of a user till the payment of an item. This is one test case out of many such cases.

3) Outline goals, business objectives, and strategic fit.

The team goals and business objectives section are meant to give you a chance to add some color and context to your requirement.

A good way to think about your goals is from your customer’s perspective. What exactly are they trying to achieve? How does the solution you’re proposing fit into it?

4) Creation of User Stories.

When all of the requirements of what to test are clearly agreed by all the team we now need to turn this into a story board that captures the User Story’s in the best possible way. See below how we have created this in Jira and then mapped it to our test requirements.

In the above screenshot, you can see some examples of our User Stories.

Let’s walk you through how we created these user story’s one by one:-

1) If We need user registration of 10 Users on AutomationPractice.com

Then we can simply create user story such as:- User Registration for 10 Users on AutomationPractice.com

2) If we want a user without login to be able to place an order on AutomationPractice.com

Then we can simply create user story such as:- Verify As a Guest User, A User should be able to search T-shirt in Search Box and proceed to Checkout Page

3) If we want a user to be able to subscribe to our newsletter on AutomationPractice.com to get information about the latest products.

Then we can simply create user story such as:- Verify User should be able to Subscribe to the WebSite Newsletter.

4) If we want that a Registered user to be able to order an Item through completing the payment of that item.

Then we can simply create user stories such as:- Verify as a Registered User, A user should be able to search Printed Dress in Search Box and able to Add it into his shopping cart and complete the payment process.

Likewise, we can create many more user stories like this on Jira by mapping the test requirements to user stories.

Before moving to AI-Driven Continous Test Automation, let’s first look at how we would use one of the conventional test frameworks such as Mocha. 

Mocha is a feature-rich Javascript test framework running on Node.js and in the browser, making asynchronous testing simple and productive. Mocha tests run serially, allowing for flexible and accurate reporting while mapping uncaught exceptions to the correct test cases.

There are many conventional test environments like Mocha used widely today including Jasmine, Jest, and Espresso.

All of these approaches require the user to write many individual tests that are then tied together to execute test scenarios. The user focuses their test writing on a very directed user-journey and one-shot data used.

Running these tests typically takes minutes and to achieve high test coverage the user will need to write test after test after test. Using their intuition and knowledge of their app under test many weeks of test scripting and debugging will be undertaken.

There is a more efficient and effective way of achieving high test space coverage by using the power of variability and the compute of AI-Driven to reduce the amount of scriptwriting needed and maximize the continuous test execution of data-driven test scenarios to achieve much more testing with much less effort. 

But before we introduce you to your new life as a Tester. Let’s briefly touch on what we do today with conventional test frameworks like Mocha. 

After Gathering Requirements we usually create a Test Environment. We have used Page-Object-Model for that. As we know page-object-model approach allows you to split your test logic from the UI representation.

As a result:-

1. You can develop your tests faster

2. You can re-use earlier created objects for one test in another test

3. If UI changes you don’t need to change your test logic, all you need to do is to amend the relevant Page Object(s) to reflect the changes – the test code remains the same

Following Page-Object-Model, we have also created a sort of framework for carrying out Test Activities on AutomationPractice.com

Here we have kept page objects for every page under the folder named as pages:-

Likewise, we have kept our test files under the Test Folder.

We of course have a config file that checks which test we need to run and on which browser.

This is the Conventional Approach we are following in our day to day Test Automation Task, writing test after test after test after test. Are you exhausted yet?

How to map your JavaScript Test framework code from Mocha, Jasmine, Jest or Espresso into Vitaq

Vitaq AI works seamlessly with Selenium, Appium, visual coding tools like Scriptworks, all CI tools, visual checking tools like Applitools and all cloud device platforms such as Saucelabs. At present this AI-driven test Automation Tool is Delivered in Docker for Mac, Windows and Linux. This product has been developed to increase UI and API test creation productivity, provide continuous auto-generated tests using the power of AI machine learning algorithms and find software functional bugs that other approaches miss. 

Inshort we can say that Vitaq AI Provides an AI-Driven, Coverage-led, continuous autonomous testing to than can be used with JavaScript, Python or Visual Code.

Mapping Mocha (or any other Test Framework) Code into Vitaq AI Test Automation Tool

Software Testers write their Automation code in their favourite IDE’s such as Eclipse, Intellij, Visual Studio Code etc. They quickly become productive in debugging their code in these IDE’s due to their feature-rich syntax checking, code completion and debugging capabilities. QA and dev testers have their favourite IDE which they gain experience with over time since they have started coding. So we have made sure that our Vitaq AI users can keep this productivity by being able to use their IDE of choice to create, check and debug their code as they create it for Vitaq AI driven test automation.

Before mapping our existing Test Framework code such as Mocha into the Vitaq-AI Test Automation Tool, We need to understand Vitaq Test Actions and Test Activity Diagrams to know what they are and how they work to help the QA and dev tester make the transition to continuous autonomous testing.

Test Actions :- Vitaq Test Actions typically represent web pages or app screens and the test scripts in these actions implement what the user can do, such as click a button or enter some data. The test data and sequences of Test Actions are auto-generated by Vitaq to efficiently fill the test space.

Test Activity Diagram :- The Test Activity Diagram in VitaqAI Test Automation Tool is a highly visual approach for describing test scenarios which ultimately makes it easier to develop and automate the generation of intelligent tests for dynamically generated content in websites and mobile apps. Many call this type of approach Model Driven testing. A Complete Test Activity Diagram consists of a number of Test Actions which are connected to their next allowable Test Actions to model what the user wants to test in a particular user journey or use-case scenario based on their test requirements. 

Let’s now take a look at how Mocha Tests can be picked up and dropped into the Vitaq script section of a particular Test Action :- 

1. Double Click on your desired Test Action for which you want to copy a Mocha JavaScript test. This might be a single function to complete a specific automated test or series of functions to test a whole web page. To Explain this concept we are going to use the user created “sign_In” Test Action of our Vitaq Test Activity diagram example below.

2. Go to the Sign_In function you have created in your Mocha Test.

3. We have used JS page object models for our Mocha tests. In our Mocha environment we have already created a signIn function which is present in the AutomationPracticeHome_Page js file. Therefore we can reuse that JS function from Mocha directly into Vitaq AI like this “AutomationPracticeHome_Page clickOnSignInLink()” inside the Vitaq sign_In Test Action script.

It is very easy to reuse all of your standard test framework code in the Vitaq AI Test Activity diagram.

Let’s now take a look at how your re-used Mocha test code can be enhanced to become data driven for highly variable exploratory tests by using Vitaq Test Activity Variables :-

In the above example we have used the sign_In Test Action which contains a simple click function which did not require any test data. When writing test code for Mocha functions which requires test data, we would typically put in fixed values. If we want highly variable data that continuously changes according to the needs of achieving coverage requirements of our test space then how can we provide that? Luckily Vitaq has come up with an ingeniously simple way of replacing our fixed date in Mocha (or other common JS framework tests) with something they call a Vitaq Test Activity Variable. 

Test Activity Variable:- Vitaq Test Activity Variables are the test script variables which hold test data and hence we can substitute them into our reused Mocha test code. In the Vitaq Test Action scripts we can call upon this test data dynamically as Vitaq AI runs. So when Vitaq AI running, is executing each Test Action Script in each Vitaq Test Action and then chooses what is the appropriate next allowable test action in the Test Activity diagram to execute next. All these next Test Action decisions and test data generation decisions are driven by the AI machine learning algorithms of Vitaq to “cover” the user journey requirements. Simply giving you continuous auto-generated AI-driven and coverage-led test scenarios.

Let’s look at the user journey example for User Registration. To test it  completely we will need test data created for the user’s first name, last name, address and phone number. We could stick to a very directed and static value based test data from Mocha or we can release the Power of variability by using the Vitaq Test Activity Variables provided by Vitaq AI Test Automation Tool to automatically create the data needed driven by the AI machine learning algorithms to the tool.

In the example for User Registration, we need the user’s first name , last name , address and phone number. All of this data can be kept inside the Test Activity Variables provided by VitaqAI Test Automation Tool.

Let’s go through the details of newsletter Test Action where the re-used Mocha code is now in the Vitaq test Action script requires newsLetterName Test Activity Variables for subscribing to newsletter of AutomationPractice.com. Let’s take a more detailed look at that to Understand it.

1) Double click on the newsletter Test Action where we are using the Vitaq Test Activity Variable named as newsLetterName Present under Activity Variable Tab on the Right Most Side After Action Properties Tab.

2) Inside the newsletter Test Action Script, we can see Test Activity Variable newsLetterName has been called using requestData() which is one of the Vitaq AI library methods which we can simply use to generate and return new values for our defined Vitaq Test Activity Variables when they are called in a Test Action Script.

Each time Vitaq AI runs this Test Action and executes the JS Test Action Script, Vitaq AI will decide what are the most appropriate data values to create for the defined Test Activity Variables and hence dynamically enhances your reused Mocha test action scripts for Data Driven Testing.

Test Activity Log and Script Output in VitaqAI Test Automation Tool

The Vitaq AI Test Automation Tool helps you to debug your Test Activity Diagrams when your test runs show failures in User Journeys or Use Case Sequences. The two main debug windows are Test Activity Log and Script Output tab.

Before we jump into Test Activity Log and Script Output Tab let’s quickly highlight what debugging is and why it is important to you as QA tester.

In general, we think about debugging only when a problem comes up while writing and executing our test code and we do not know how that problem has occurred. For diagnosing defects and problems in software code you will always need a good debugger which is why you should get to know your IDE well.

Let’s discuss some of the facts to frame the discussion :-

1) The later the bug is found in the software development process, the costlier and harder it is to fix it. If the code written by the developer is buggy and we can find out about it in the earlier stage of development then its easier to fix it before it becomes a critical component of our customers interface to their customers. But if it gets caught at a later stage of time then many newly developed modules are related to it and that can cause major restructuring. Worse yet, if the bug is discovered after the product has shipped, it may require recalling the entire product. In short, the sooner the bug is found easier it is to fix it.

2) Studies have shown that there is 1 bug in every 10 lines of code and that can only be discovered when that application is tested by hitting the edge conditions.

3) Other research on testing shows that 55% to 70% of the time spent on a typical software project is spent on testing and debugging. This large amount of time spent on testing and debugging makes it very hard to schedule projects accurately and is a significant cost burden for any software based product.

4) Software development methodologies around the world have attempted to address these issues by reducing the number of bugs that get built into the product in the first place.

We all know that every code could have probability of having bug in it, the sooner we can eliminate these bugs, the fewer problems will need to be fixed in the testing and debugging phase and the sooner our project can be released

How Test Activity Log and Script Output in Vitaq AI Test Automation Tool help in Debugging

But what if we create bugs in our test code in Vitaq? Let’s say while running our Test Activity Diagram we face several issues that might occur due to error in software under test code or our test code or any specific environment issues then how can we debug?

Here comes Test Activity Log and Script Output in Picture. Let’s see a few common issues and try to debug them via Test Activity Log and Script Output messages.

1) When there is a mis-match between naming convention of defined Test Action and Function/Method present inside the Test Action.

2) When user somehow closes window intentionally/during code execution on Vitaq AI.

3) When user fetch/request data from Test Activity Variable.

4) When there is a syntax error in any of the test action.

Likewise Test Activity Log and Script Output helps a lot in debugging the errors productively while writing and executing our Vitaq AI Test automation code.

The best methodology to follow is to use the Test Activity Log to give us the high level detail and then the script output tab can be used to do a deep dive into the error details so that we can get to understand the cause of errors.