[wp-trac] [WordPress Trac] #62171: Test suite: Eliminate flakiness from basic HTTP tests getting rate-limited.

[WordPress Trac]

#62171: Test suite: Eliminate flakiness from basic HTTP tests getting rate-limited.
-------------------------+----------------------------
 Reporter:  dmsnell      |      Owner:  (none)
     Type:  enhancement  |     Status:  new
 Priority:  normal       |  Milestone:  Future Release
Component:  General      |    Version:  trunk
 Severity:  normal       |   Keywords:
  Focuses:               |
-------------------------+----------------------------
 A number of tests in the `Tests_HTTP_Functions` class, such as
 `test_head_404`, `test_head_redirect`, and `test_get_response_cookies`
 make basic HTTP calls to `w.org`, `wp.org`, and `wordpress.org` to verify
 WordPress' core HTTP handling.

 However, these requests frequently get rate-limited, possibly because they
 give the appearance of improper activity. For example, when running a
 matrix of PHPUnit tests, a single IP address from the Github Actions
 container makes repeated requests for `/screenshots/3.9/awefasdfawef.jpg`
 - a resource that doesn't exist. The test asserts that the response is a
 404.

 Besides introducing conflating effects and network latency, these tests
 are inherently fragile because if that resource ever becomes available it
 will cause the tests to fail even though the code under test hasn't
 failed. The tests are indirect, measuring different code than they claim
 to, and asserting that the failure of the actual system under test
 represents a failure of the purported code under test.

 These flakey tests frequently block and reject working, good, and positive
 contributions to the project, so we would benefit by updating the tests to
 avoid this.

 One possible approach would be to spool up a web server inside the test
 suite, perhaps as a separate OS process, perhaps as a separate node in the
 container, which provides a method to deterministically respond as
 expected.


 {{{
 GET /index.php?status_code=404 HTTP/1.1

 HTTP/1.1 404 Not Found
 }}}

 In [https://wordpress.slack.com/archives/C02RQBWTW/p1728143324005749 the
 original Slack discussion] it was also proposed to reuse existing test
 servers, such as:

  - The server which `Requests` uses
 [https://packagist.org/packages/requests/test-server test-server]
  - An existing PHP package like [https://github.com/donatj/mock-webserver
 mock-webserver]
  - An incredibly basic PHP file served via `php -S` that only provides
 what the tests need, and is fully controlled in this repository.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/62171>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform