[wp-trac] [WordPress Trac] #53119: Tests: introduce naming conventions for data providers and use named test cases

[WordPress Trac]

#53119: Tests: introduce naming conventions for data providers and use named test
cases
------------------------------+------------------------------
 Reporter:  jrf               |       Owner:  (none)
     Type:  task (blessed)    |      Status:  new
 Priority:  normal            |   Milestone:  Awaiting Review
Component:  Build/Test Tools  |     Version:
 Severity:  normal            |  Resolution:
 Keywords:                    |     Focuses:  docs
------------------------------+------------------------------
Description changed by jrf:

Old description:

> A number of tests use dataproviders - functions which return a multi-
> level array of test cases, where each sub-array is a set of parameters to
> pass to the actual test function -.
>
> Also see: https://phpunit.readthedocs.io/en/9.5/writing-tests-for-
> phpunit.html?#data-providers
>
> Currently there are:
> * No naming conventions for data providers - the functions are called
> whatever the author wanted to call them.
> * No placement conventions for data providers - sometimes the functions
> are at the top of a file, sometimes at the bottom, sometimes just before
> or after the test function using them.
>
> I'd like to recommend streamlining this a little more to:
> 1. Make it more obvious which functions are data providers.
> 2. Make the data providers easier to find.
>
> With that in mind, I'm proposing:
> 1. That the function name of all data provider functions starts with
> `data_`, similar to how all test functions are prefixed with `test_`.
> 2. That data providers when used by only one test are placed directly
> after that test.
> 3. That data providers when used by multiple tests which directly follow
> each other, are placed directly after the last test using the data
> provider.
> 4. That data providers used by multiple tests throughout a test file are
> placed at the bottom of the class.
>

> Additionally, there is an awesome, but little known feature in PHPUnit
> which allows to "name" each test case in a data provider by adding an
> index key for each test data set.
> This will make test failure information more descriptive and will make
> the test case which is causing the failure easier to find than having the
> default numeric indexes for test cases.
>
> And when coupled with the `--testdox` feature, especially when
> selectively running a filtered set of tests will make it much more
> obvious what is being tested.
>
> So I'd also like to recommend implementing the use of named test cases
> when using data providers.
>
> To illustrate the difference:
> Screenshot of a test run with a dataprovider and `--testdox` without
> using named test cases:
> https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-
> e6ea54ce-2515-4ea9-aacf-9bf7ab3b3141?slide=26
> Screenshot of a test run with a dataprovider and `--testdox` WITH named
> test cases:
> https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-
> e6ea54ce-2515-4ea9-aacf-9bf7ab3b3141?slide=28
>
> Code sample of how to implement this:
> https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-
> e6ea54ce-2515-4ea9-aacf-9bf7ab3b3141?slide=27
>

> Once this ticket has been actioned, I'd like to recommend that the
> conventions applied will be added to the Core contributors handbook.
>
> /cc @hellofromTonya

New description:

 A number of tests use dataproviders - functions which return a multi-level
 array of test cases, where each sub-array is a set of parameters to pass
 to the actual test function -.

 Also see: https://phpunit.readthedocs.io/en/9.5/writing-tests-for-
 phpunit.html?#data-providers

 Currently there are:
 * No naming conventions for data providers - the functions are called
 whatever the author wanted to call them.
 * No placement conventions for data providers - sometimes the functions
 are at the top of a file, sometimes at the bottom, sometimes just before
 or after the test function using them.

 I'd like to recommend streamlining this a little more to:
 1. Make it more obvious which functions are data providers.
 2. Make the data providers easier to find.

 With that in mind, I'm proposing:
 1. That the function name of all data provider functions starts with
 `data_`, similar to how all test functions are prefixed with `test_`.
 2. That data providers when used by only one test are placed directly
 after that test.
 3. That data providers when used by multiple tests which directly follow
 each other, are placed directly after the last test using the data
 provider.
 4. ~~That data providers used by multiple tests throughout a test file are
 placed at the bottom of the class.~~ **Updated**: That when a data
 provider is used by multiple tests throughout a test file, that the tests
 using the data provider are grouped together and that the data provider is
 then placed directly after the last of the tests using it, as per 3.


 Additionally, there is an awesome, but little known feature in PHPUnit
 which allows to "name" each test case in a data provider by adding an
 index key for each test data set.
 This will make test failure information more descriptive and will make the
 test case which is causing the failure easier to find than having the
 default numeric indexes for test cases.

 And when coupled with the `--testdox` feature, especially when selectively
 running a filtered set of tests will make it much more obvious what is
 being tested.

 So I'd also like to recommend implementing the use of named test cases
 when using data providers.

 To illustrate the difference:
 Screenshot of a test run with a dataprovider and `--testdox` without using
 named test cases:
 https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-
 e6ea54ce-2515-4ea9-aacf-9bf7ab3b3141?slide=26
 Screenshot of a test run with a dataprovider and `--testdox` WITH named
 test cases:
 https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-
 e6ea54ce-2515-4ea9-aacf-9bf7ab3b3141?slide=28

 Code sample of how to implement this:
 https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-
 e6ea54ce-2515-4ea9-aacf-9bf7ab3b3141?slide=27


 Once this ticket has been actioned, I'd like to recommend that the
 conventions applied will be added to the Core contributors handbook.

 /cc @hellofromTonya

--

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/53119#comment:9>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform