Running the Magento 2 Test Suite

Magento 2 ships with a complete testing suite out-of-the-box. These include a complete set of operational unit, integration, and static tests. The simplest of these to run is probably the unit tests, having all necessary components setup and in-place automatically after installing Magento 2 successfully. Here I’ll show you how to run the full tests suite, run each type of tests independently, and also how to run a small portion of the unit or integration tests if need be. Before starting, you’ll need to ensure you have a working copy of Magento 2 installed and running successfully. Instructions on how to do that can be found here in my post on Installing Magento 2.

Note: The Magento Testing Framework does support functional testing, but that goes beyond the scope of this post and may be covered in a future writeup.

To power the test suite, Magento uses phpunit, with the test suite configuration residing under ./dev/ in your installation of Magento 2. In the following tree you’ll notice a few directories. The three named unit, integration, and static are where you’ll be working from to run the tests.

$ tree -d -L 1 dev/tests/
dev/tests/
├── api-functional
├── functional
├── integration
├── js
├── static
└── unit

6 directories

Since Magento 2 uses phpunit to run it’s test suite, you will need to either install this tool in your environment globally or run the copy composer installs at vendor/phpunit/phpunit/phpunit. For the examples below, I’ll be using the phpunit installed by composer in the project directory.

Running Unit Tests

These are simplest tests to run and they require no configuration on your part beyond ensuring you have a working copy of phpunit available. To run just the unit tests:

cd dev/tests/unit/
../../../vendor/phpunit/phpunit/phpunit

To run an individual set of tests, like say all the unit tests for the Magento_Catalog module:

cd dev/tests/unit/
../../../vendor/phpunit/phpunit/phpunit \
    ../../../vendor/magento/module-catalog/Test/Unit/

You can even pair this down further and run only the tests in a single file:

cd dev/tests/unit/
../../../vendor/phpunit/phpunit/phpunit \
    ../../../vendor/magento/module-catalog/Test/Unit/Model/Product/ImageTest.php

The following is the output of a successful run from this last example:

PHPUnit 4.1.0 by Sebastian Bergmann.

Configuration read from /sites/m2.demo/dev/tests/unit/phpunit.xml.dist

........................

Time: 621 ms, Memory: 12.00Mb

OK (24 tests, 86 assertions)

Running Integration Tests

Getting integration tests up and running takes a little bit of extra work. This is because for these tests to run, they have to standup a copy of Magento 2 for tests purposes complete with a working database to query against. Here’s how to do this:

Step 1: Place the following (updating it with proper credentials for your MySql server) in dev/tests/integration/etc/install-config-mysql.php:

<?php

return [
    'db-host' => 'dev-db',
    'db-user' => 'root',
    'db-password' => '',
    'db-name' => 'magento_integration_tests',
    'db-prefix' => '',
    'backend-frontname' => 'backend',
    'admin-user' => \Magento\TestFramework\Bootstrap::ADMIN_NAME,
    'admin-password' => \Magento\TestFramework\Bootstrap::ADMIN_PASSWORD,
    'admin-email' => \Magento\TestFramework\Bootstrap::ADMIN_EMAIL,
    'admin-firstname' => \Magento\TestFramework\Bootstrap::ADMIN_FIRSTNAME,
    'admin-lastname' => \Magento\TestFramework\Bootstrap::ADMIN_LASTNAME,
];

Step 2: Create the database referenced in the above config:

mysql -e 'create database magento_integration_tests'

Step 3: Use the following to run the integration tests:

cd dev/tests/integration/
../../../vendor/phpunit/phpunit/phpunit

You can, as with the unit tests, choose to run a sub-set of the integration tests. Because these are global to application, all tests are found within this dev/tests/integration location. For example:

cd dev/tests/integration/
../../../vendor/phpunit/phpunit/phpunit \
    testsuite/Magento/Catalog/Model/Product/ImageTest.php

Here is the output from a successful run of this last example:

PHPUnit 4.1.0 by Sebastian Bergmann.

Configuration read from /sites/m2.demo/dev/tests/integration/phpunit.xml.dist

....

Time: 1.1 minutes, Memory: 409.75Mb

OK (4 tests, 5 assertions)

=== Memory Usage System Stats ===
Memory usage (OS):	430.38M (105.16% of 409.25M reported by PHP)
Estimated memory leak:	21.13M (4.91% of used memory)

Running Static Tests

These are also fairly simple to run, but they are broken into two chunks:

cd dev/tests/static/
../../../vendor/phpunit/phpunit/phpunit

…and this…

cd dev/tests/static/framework/tests/unit
../../../../../../vendor/phpunit/phpunit/phpunit

Running The Complete Suite

Assuming you have setup the configuration for integration tests as noted above, running the complete tests suite is fairly trivial, although doing so can take an incredible amount of time to run. From the root of your installation:

bin/magento dev:tests:run

This handy command built into the CLI tool will run through the unit tests, the static-framework tests, integration tests, and the remaining static tests (in that order). When complete, you’ll see summarized out put of the results from the run.

Have fun testing!