HomePhorge

Add `phpunit:prepare-parallel:extensions` command

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

Add phpunit:prepare-parallel:extensions command

In T361190 and Quibble 1.9.0, we introduced parallel execution of
PHPUnit tests to speed up the CI jobs. The existing implementation
is purely Python/Quibble, and cannot directly be used by developers
locally. With this patch, we re-implement the test splitting logic
already implemented in CI as a composer task so that the parallel
tests can be run locally.

There are a couple of different approaches to running PHPUnit tests
in parallel. The different approaches have been discussed at length
in T50217. Ideally, we would just install the paratest extension
and use that to parallelise the execution. Unfortunately we have
complex test suites (specifically Parser tests and the Scribunto
test suite) that dynamically create tests as they run, which makes
it hard for paratest to work out which tests will run.

To overcome this limitation, we use the phpunit --list-tests
function to create a list of test classes that would be included in
the execution of the test suite, then scan the filesystem for
classes named in the tests-list.xml output. The classes we find
are then collected into smaller groups (split_group_X) which we
can run in parallel in separate processes.

We split into 7-8 groups here, as that experimentally leads to an
even spread of the tests and consumes 100% of all cores on a 4-core
processor.

Because ParserIntegrationTest.php is a single test class that
generates thousands of integration tests, we put that in its own
bucket rather than allocating it round-robin to one of the split
buckets. This again helps to keep the buckets roughly the same size.

The current implementation only supports splitting the extensions
test suite. We need to do some more development and testing to
support splitting other suites.

The new composer command phpunit:prepare-parallel:extensions will
generate a phpunit.xml file with the same contents as
phpunit.xml.dist, but with the split-group suites added. The
result of running all of the split groups should be the same as the
result of running the whole test suite.

Bug: T365976
Change-Id: I2d841ab236c5367961603bb526319053551bec2e

Details

Provenance
Arthur TaylorAuthored on May 31 2024, 12:44 PM
Kosta HarlanCommitted on Jul 10 2024, 7:59 AM

Event Timeline

Kosta Harlan <kharlan@wikimedia.org> committed rMW1e2851d8d160: Add `phpunit:prepare-parallel:extensions` command (authored by Arthur Taylor <arthur.taylor@wikimedia.de>).Jul 10 2024, 7:59 AM

Commit No Longer Exists

This commit no longer exists in the repository.