Skip to main content

Getting started with Vitaq AI

To get up and running with Vitaq AI as quickly as possible we would suggest that you do the following:

  1. Create a Vitaq account, login and follow the tour to get a quick understanding of the example.
  2. Run the provided example for yourself.
  3. Create your own diagram and run it and start using Vitaq.

These are brief descriptions of the steps you need to follow to get going with links to more detailed information for when you need to go deeper. At the end of each are some suggested things to try or look at to help you quickly understand what's going on.

Prerequisites​

For these examples it is assumed that you have Node.js and NPM installed and that you can run them at the command line in a terminal.

You may also want to have an Integrated Development Environment (IDE) installed such as Visual Studio Code, this will help you in viewing and editing the action scripts and other files used.

These examples are designed to work with the Google Chrome browser and it is assumed that you have it installed. If not, then you can download it from here.

Finally, you will need to have created a Vitaq account with a registered e-mail address and password. You can register here.

Get started with the provided example​

  1. On your local machine create a new directory and cd into it.
mkdir vitaq-example
cd vitaq-example
  1. Initialise a new NPM project, and install WebdriverIO CLI (See more ...)
npm init -y
npm install @wdio/cli
  1. Install the custom Vitaq modules to work with WebdriverIO (See more ...)
npm install wdio-vitaqai-service
  1. Run the Vitaq example script and answer the questions. This script will login to your Vitaq account and download the example data, which it will unpack into the local directory. The script will also customise the WebdriverIO configuration file (wdio.conf.js) so that it can work with Vitaq.
npx vitaq_example
*Note:  This step will install the latest npm chromedriver package. If you have an older version of Chrome installed, this may not be the appropriate version to use, and you may get an error of the form:

WARN webdriver: Request failed with status 500 due to session not created: This version of ChromeDriver only supports Chrome version 109

You can either upgrade your version of Chrome or install the chromedirver version which matches your version of Chrome using:

npm install chromedriver@xx.0

where xx is the major version number of Chrome e.g. 109

  1. Run the Vitaq example. This step will launch WebdriverIO and an instance of Chrome and open the Sauce Demo website. Vitaq will select the sequence of test actions to run, based on the onBoarding->swaglabs test activity diagram which is provided as an example in your account. The commands that are executed for each test action are contained in the files with matching names in the test/actions folder.
npx wdio --useAI

This will run Vitaq AI in AI mode. Vitaq AI will select which test actions to execute based on the functional coverage targets that are specified in the test activity diagram for the sequences, actions and variables. The test will stop when all of the functional coverage targets have been met. From a single diagram, we have been able to define and execute multiple different user journeys.

So, what just happened?​

Hopefully, you just saw a web browser open and run through a series of tests and then stop, but what just happened and what are the advantages of testing this way?

What just happened ...​

When you ran the command:

npx wdio --useAI

The first thing to note is that the spec files which are in the test/actions directory are not the usual end-to-end spec files that would normally be used with WebdriverIO. Instead they are spec files which correlate directly to the actions on the test activity diagram. They would typically contain the testing to perform on a single page, but they don't have to be and you can structure it how you want. The only requirement is that we can link them together as shown in the test activity diagram.

The WebdriverIO Vitaq service creates and runs a Vitaq test script in the cloud. The test script is created from the test activity diagram which is specified in the wdio.conf.js file. Webdriverio launches the Mocha test framework as it normally would, but instead of just running through all of the spec files in turn it sends a request to the Vitaq service to ask which action (or spec file) to run next. Vitaq will select the next action:

  • simply by following the list of actions as defined by a sequence or sequences (using the --sequence or --sequences command line switch). This is directed testing and is closest to the normal way of using WebdriverIO OR
  • using the machine learning algorithm to determine which action to select next in order to meet the functional coverage targets that are defined in the test activity diagram on the sequences, the actions and the variables OR
  • by making a random selection based on various rules (or constraints) Vitaq sends the name of the next action to WebdriverIO and it is executed and the result returned to Vitaq where it is recorded in the run history and the next action selected. The run continues like this until Vitaq reaches a point where there is no next action. It is worth emphasising at this point that this means that some spec files could be executed multiple times. It becomes very easy to create tests that will randomly walk through your app in the same way that a user would.

... and why would you want to do this?​

There are multiple advantages to this style of testing:

  1. When we draw the test activity diagram we are defining what is allowable in the app. This is very different to the normal way of writing a test where we are normally trying to think of all of the things that could happen and write a test for them. By defining what is allowable and then allowing Vitaq to randomly select which next action to go to we get sequences of actions that we would not have thought of and consequently find bugs that the directed approach would miss. However, this does not mean that we are just hoping that we will test areas that might cause concern. Because we can specify sequences we can easily define the typical user journeys we need to see tested. As a result of this approach we get a lot of extra testing for no extra effort.

  2. In all but the simplest of apps there are multiple paths that can be taken through the app, especially if there is any aspect of looping, such as going to a product page, looking at an item, adding the item to a cart and then going back to the product page. Because Vitaq uses the test activity diagram, all of these paths are defined and a single diagram defines all of the tests for a given app.

  3. If the app changes, for example a page is added or removed, it is very simple to modify the test activity diagram to reflect the changes. We don't have to go through all of the tests and change them. Essentially we have made the testing for each page modular and they can be plugged/unplugged from the testing at will.

  4. Since each spec file corresponds to an action, any changes to the testing code in the spec file will immediately be executed by every test that runs that action.

  5. Each of the test actions has a set of properties that can be set. For example we can enable and disable actions, meaning we can add/remove actions from the testing without significant modification. We can set a call limit on an action so it can only be executed a certain number of times and we can specify a coverage target which causes the AI to ensure that the action is run at least that many times.

  6. Vitaq has an API that allows us to interact with the diagram from the test code. So, for example we can set actions enabled or disabled through the spec file or reset the call count or change the call limit. This means that tests can become very dynamic depending on what is happening in the app and gives you the flexibility to write very powerful tests. In addition to interacting with the diagram, the API allows us to interact with the test variables, which can be used in the spec files.

  7. As you will have seen on the tour, Vitaq collects functional coverage data for the sequences, the actions and the variables.

  8. Vitaq is based on random selection of the next action for at least some of the time, unless it is following a sequence. However, Vitaq has what we call random stability, meaning that if we use the same seed for a second time we will get exactly the same test (assuming nothing else has changed).

Suggestions for digging a little bit deeper to get familiar with Vitaq​

Get started with your own diagram​

  1. Create a test activity diagram
  1. On your local machine create a new directory and cd into it.
mkdir vitaq-quickstart
cd vitaq-quickstart
  1. Initialise a new NPM project, and install WebdriverIO CLI. (See more ...)
npm init -y
npm install @wdio/cli
  1. Install the custom Vitaq modules to work with WebdriverIO (See more ...)
npm install wdio-vitaqai-service
  1. Run the Vitaq quickstart script and answer the questions. This will create the directory structure and populate the test/actions directory with boilerplate scripts, with a script for each of the test actions in your test activity diagram. It will also create a WebdriverIO configuration file wdio.conf.js, configured to allow Vitaq to run.
npx vitaq_quickstart
  1. Run your first Vitaq test activity. This step will launch WebdriverIO and an instance of Chrome and open the Quickstart Home page. Vitaq will select the sequence of test actions to run, based on the test activity diagram you created. The commands that are executed for each test action are contained in the files with matching names in the test/actions folder. The boilerplate scripts allow WebdriverIO to run, but they don't do anything specific. You will need to customise the scripts for testing your target website, follow the instructions on the Quickstart Home page.
npx wdio
*Note:  The vitaq_quickstart utility can be used anytime you create a new diagram or add actions to an existing diagram to automatically create/update the directory structure and boilerplate code. See more details in the Vitaq Utility Scripts section.

So, what just happened?​

Hopefully you will have seen Chrome launch and open the quickstart page. As already mentioned above, the scripts in test\actions are just example scripts that don't really do anything. It is now over to you to specify the website you wish to test and start writing the code to test it. If you need help or have a question or you are just completely lost, please contact us and we will be happy to help.

Suggestions for digging a little bit deeper ...​

npx wdio --useAI --useCoverage

Install WebdriverIO with VitaqAI-Mocha​

Vitaq AI works on top of WebDriverIO. WebdriverIO is a next-generation browser and mobile automation test framework. It is written in JavaScript and is not restricted to just Selenium or Appium because it supports the Chrome Devtools protocol directly. Which means that you can get started without having to install Java or Selenium.

You write your tests in the language of front-end developers, JavaScript. With the same testing tools that you are used to using for unit tests, Mocha. The commands you use in your WebdriverIO tests are concise and common sense.

WebdriverIO is a Node.js based system, so the first thing you need to do is install Node (if you do not have it installed already). Go to https://nodejs.org/en/ and follow the instructions or install it using a version manager such as NVM https://github.com/nvm-sh/nvm.

The installation instructions for WebDriverIO can be found here https://webdriver.io/docs/gettingstarted/

First set up a new folder for writing tests in. In the example below we will call it wdio-vitaq (you can name it what you like).

mkdir wdio-vitaq
cd wdio-vitaq

From inside that folder initialise a new NPM project

npm init -y

That will create a package.json file that is used for saving dependencies.

Now you can install the WDIO CLI ("Command Line Interface")

npm install @wdio/cli

Installing ChromeDriver​

To get started as quickly as possible we are not going to install Selenium or Appium. We are going to show you how to test a web App in a Chrome Browser using the ChromeDriver.

npm install -g chromedriver

Setting Up WebdriverIO​

To setup WebdriverIO with the base configuration, run:

npx wdio config --yes
IMPORTANT NOTE: Vitaq uses a modified version of the Mocha framework which we distribute in a package called vitaqai-mocha. This package will sit alongside your existing Mocha installation and will be specified by configuration settings as described below. The vitaqai-mocha package is installed as a dependency of the wdio-vitaqai-service package. Vitaq does not yet support Jest or Cucumber, so you will need to write your TDD/BDD tests in Mocha.
npm install wdio-vitaqai-service

You will also need to edit the default wdio.conf.js file that was automatically created in this directory to set the test specs to be grouped, add in the vitaqai service and define the framework to use as vitaqai-mocha.

"specs" section​

The definition of the specs in the WDIO config file has been extended so that it can now accept arrays within the specs array. All of the files within an inner array [['VitaqAI Spec action .js files here']] are grouped together and run in the same instance.

We suggest you keep your Vitaq WebdriverIO Spec Action scripts in a directory called actions underneath the WebdriverIO test directory. So then you configure your wdio.conf.js with:

specs: [
        ['./test/actions/**.js']
],

"services" section​

To add in the vitaqai service you need to modify your wdio.conf.js as shown below, if you used the example or the quickstart scripts then this will have been done for you. You can see details of the available options here. 

services: [
    ['vitaqai', {
        userName: 'your-user-name',
        testActivityName: 'swaglabs',
        projectName: "onBoarding",
        url: "https://vitaq.online",
        userAPIKey: "get this long key from your vitaq.online account and paste here"
        ,
        useSync: false
    }],
    'chromedriver',
],

"framework" section​

Vitaq only supports the modified version of the Mocha framework, so you need to define this framework here:

framework: 'vitaqai-mocha'

Creating a Vitaq AI Test Activity to auto-generate tests​

This flow diagram provides a high-level view of the steps needed to create your first Test Activity with Vitaq AI to auto-generate tests with webDriverIO.

Logging into Vitaq AI​

Go to https://vitaq.online and click on the Login button. Complete login with your provided username and password. Create a Project by completing the project Name field and clicking Submit. Click on your created Project and then click on Create Test Activity. Complete the Test Activity name field and then click on Submit. Click on your created Test Activity Name and the Vitaq AI Test Activity editor will launch.

You will notice that your account is preloaded with the onBoarding project which contains the swaglabs Test Activity. We will use this to show you how to run your first end-to-end tests with Vitaq AI.

Running your first Test Activity​

In the directory you created for writing tests (in which we recommended you call wdio-vitaq above) you can run your Test Activity by using the 'run' command and pointing to the WebdriverIO config that you just created.

Now you can run the swaglabs Test Activity listed in testActivityName in the config file.

>npx wdio run  wdio.conf.js

To run the Test Activity using the AI-driven capability, where Vitaq will "machine-learn" what next allowable tests it needs to auto-generate to reach 100% test coverage use these options.

>npx wdio run  wdio.conf.js --useCoverage true --useAI true

Once completed, you should see that the WebdriverIO tests passed. Then you can open the swaglabs Test Activity in your browser, click on the coverage icon and open the functional coverage results for the Test Activity run to see how many end-to-end tests were created to complete 100% user journey coverage.