Automated Testing

Introduction

Automated testing provides a means to ensure that the high quality of SuiteCRM is maintained. SuiteCRM has an automated test suite which is powered by the Codeception and PHPUnit testing frameworks.

Test Suites

SuiteCRM offers four different test suites.

Suite Description

install

Acceptance test verify that the install wizard is working correctly

acceptance

Automated browser testing, also known as 'feature tests'

unit

Unit tests verify that a single unit of code, e.g. a method, is working correctly

api

Functional tests which test the API version 8 responses

Setup

This guide assumes you have already installed and set up a supported PHP version, MySQL 5.7, and Composer.

You can set up the CRM for testing with these steps:

  • In your terminal emulator or command prompt: cd path/to/suitecrm/instance

  • Install dependencies with Composer: composer install

  • Create a MySQL database called automated_tests, along with a user called automated_tests and the password automated_tests. In the MySQL command line interface:

  • mysql -e "CREATE DATABASE automated_tests CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;"

  • mysql -u root -p -e "CREATE USER 'automated_tests'@'localhost' IDENTIFIED BY 'automated_tests';"

  • mysql -u root -p -e "GRANT ALL PRIVILEGES ON automated_tests.* TO 'automated_tests'@'localhost';"

  • Configure environment variables, see environment variables section below for more information.

  • Configure your Apache server so the SuiteCRM server is available at http://localhost (or whatever instance URL you’ve set up in the previous step).

  • Install ChromeDriver so you can use it to automate the Chrome browser (you’ll also need Chrome or Chromium installed): ./vendor/bin/robo chromedriver:install

  • You’ll also need an instance of ChromeDriver running in the background for the test suite to use, you can use ./vendor/bin/robo chromedriver:run to run a ChromeDriver instance. This is necessary for the install and acceptance suites.

  • You can update your ChromeDriver version if necessary with ./vendor/bin/robo chromedriver:install --reinstall.

  • Build test dependencies with ./vendor/bin/codecept build.

  • Set up the custom codeception environment to enable running the install and acceptance tests locally.

  • See "Configuring the test environment" below.

  • Install SuiteCRM.

  • This can be done either by running the install suite or by going through the install process in your browser. It should be available at http://localhost/install.php.

  • Populate the database with test data for the API test suite. Replace DATABASE_NAME with your configured MySQL database’s name, probably automated_tests if you followed the instructions above.

  • mysql -u root -p -D DATABASE_NAME -v -e "source tests/_data/api_data.sql"

  • mysql -u root -p -D DATABASE_NAME -v -e "source tests/_data/demo_users.sql"

With that, you should be able to run the test suites - note that the install tests can only be run once currently.

Environment variables

This is the preferred method to store sensitive information, as it prevents security information from being committed to the git repository. You can automate different development environments using environment variables.

Test suite environment variables

Install Test Suite:

Variable Description

DATABASE_DRIVER

MYSQL or MSSQL

DATABASE_HOST

path to database server

DATABASE_NAME

name of the database

DATABASE_USER

database user

DATABASE_PASSWORD

database password

Acceptance, API, and Install Test Suites:

Variable Description

INSTANCE_URL

URL of the SuiteCRM instance which the tester need to access

INSTANCE_ADMIN_USER

admin user for logging in

INSTANCE_ADMIN_PASSWORD

admin password for logging in

API Test Suites:

Variable

Description

INSTANCE_CLIENT_ID

ID of the client

INSTANCE_CLIENT_SECRET

Secret of the client

Set up environment variables with Robo task

Open terminal and run Robo:

  • ./vendor/bin/robo configure:tests (Bash)

  • .\vendor\bin\robo configure:tests (Command Prompt)

This will interactively add the necessary environment variables to your ~/.bash_aliases file. You need to restart your terminal/command prompt in order to get the changes. On macOS/Linux you can use env to see your current environment variables.

Note that these variables will apply for the entire device, so if you have more than one CRM set up locally they’ll all use the same configuration.

Configuring the test environment

It’s recommended that you run the test suite with a custom environment set up, to make sure the test suite knows where the CRM instance is, and to allow you to customize the environment to support debugging if necessary. Copy tests/_envs/custom.dist.yml to tests/_envs/custom.yml and read through the file’s comments to configure your environment. custom.yml won’t be tracked by git, so it can be customized to work with your local environment without being reset or accidentally committed.

You can run codeception tests in a given environment like so: ./vendor/bin/codecept run acceptance --env custom

Note that the custom environment is only meant for use with the acceptance and install test suites. There’s also the travis-ci-hub.yml environment, which is meant for running the tests in Travis CI.

See the Codeception documentation for more information on configuring the environment.

Running the test suites

Once they’re set up, you can run the test suites with the following Robo commands:

Suite Command

install

./vendor/bin/robo tests:install

acceptance

./vendor/bin/robo tests:acceptance

unit

./vendor/bin/robo tests:unit

api

./vendor/bin/robo tests:api

A few things to note:

  • These commands will automatically use the environment defined in tests/_envs/custom.yml for the install and acceptance test suites.

  • Each of these commands take these flags:

  • --debug: Print more info during test runs, good for debugging.

  • --fail-fast: Stop running the test suite after the first failure.

  • Each of these can also be passed a specific directory or folder if you only want to run a few specific tests, e.g. ./vendor/bin/robo tests:unit ./tests/unit/phpunit/modules/.

  • The install test suite can only be run once currently, and all other test suites depend on the CRM being installed.

  • The install and acceptance test suites use automated browser testing with Chrome and require that the user runs a separate ChromeDriver process simultaneously with the test suite. Alternatively, you enable the ChromeDriver RunProcess extension provided in custom.dist.yml to have ChromeDriver boot automatically whenever the tests need it.

Other configuration options

Docker

You can also run the test suite using Docker, if you prefer.

Setup environment variables (Docker Compose):

You can add a .env file into your Docker Compose setup:

DATABASE_DRIVER=MYSQL
DATABASE_NAME=automated_tests
DATABASE_HOST=localhost
DATABASE_USER=automated_tests
DATABASE_PASSWORD=automated_tests
INSTANCE_URL=http://path/to/instance
INSTANCE_ADMIN_USER=admin
INSTANCE_ADMIN_PASSWORD=admin
INSTANCE_CLIENT_ID=suitecrm_client
INSTANCE_CLIENT_SECRET=secret

then reference it in your php container (docker-compose.yml):

version: '3'
services:
  php:
      image: php:7.0-apache
      restart: always
      ports:
        - 9001:80
      environment:
       - DATABASE_DRIVER: $DATABASE_DRIVER
       - DATABASE_NAME: $DATABASE_NAME
       - DATABASE_HOST: $DATABASE_HOST
       - DATABASE_USER: $DATABASE_USER
       - DATABASE_PASSWORD: $DATABASE_PASSWORD
       - INSTANCE_URL: $INSTANCE_URL
       - INSTANCE_ADMIN_USER: $INSTANCE_ADMIN_USER
       - INSTANCE_ADMIN_PASSWORD: $INSTANCE_ADMIN_PASSWORD
       - INSTANCE_CLIENT_ID: $INSTANCE_CLIENT_ID
       - INSTANCE_CLIENT_SECRET: $INSTANCE_CLIENT_SECRET

Using Docker Compose with the Selenium Hub

In your selenium development environment it is recommended that you employ docker compose to set up a selenium hub with a selenium node. This will ensure your version of Chrome or Firefox is kept up-to-date with the latest version. In addition, you can also run multiple versions of PHP on the same host machine.

You can configure selenium using docker compose. Please ensure you have the following in your docker-compose.yml file.

version: '3'
services:
    selenium-hub:
      image: selenium/hub
      restart: always
      ports:
        - 4444:4444
    selenium-node-chrome:
      image: selenium/node-chrome-debug
      restart: always
      ports:
        - 5900:5900
      links:
        - selenium-hub:hub
      environment:
              - "HUB_PORT_4444_TCP_ADDR=selenium-hub"
              - "HUB_PORT_4444_TCP_PORT=4444"
    selenium-node-firefox:
      image: selenium/node-firefox-debug
      restart: always
      ports:
        - 5901:5900
      links:
        - selenium-hub:hub
      environment:
              - "HUB_PORT_4444_TCP_ADDR=selenium-hub"
              - "HUB_PORT_4444_TCP_PORT=4444"

Note: you can also choose different images for the nodes, for example the nodes without vnc support

Other tips

Add vendor/bin to your PATH

This will make it easier to run codeception and the other commands which live in the vendor/bin/ directory. You can add the vendor/bin location to your PATH environment variable.

Adding vendor/bin to PATH (Bash):

export PATH=$PATH:/path/to/instance/vendor/bin

Adding vendor/bin to PATH (Command Prompt):

set PATH=%PATH%;C:\path\to\instance\vendor\bin

This allows you to call the codecept and robo commands without having to prefix the command with its location. When running either of these, you should ensure that your current working directory is the same as your SuiteCRM instance.

cd /path/to/suitecrm/instance/

codecept run acceptance or robo tests:acceptance

Content is available under GNU Free Documentation License 1.3 or later unless otherwise noted.