[wp-trac] [WordPress Trac] #60227: HTML API: Add external test suite
WordPress Trac
noreply at wordpress.org
Mon Feb 5 17:44:16 UTC 2024
#60227: HTML API: Add external test suite
--------------------------------------+---------------------
Reporter: jonsurrell | Owner: (none)
Type: enhancement | Status: new
Priority: normal | Milestone: 6.5
Component: Build/Test Tools | Version:
Severity: normal | Resolution:
Keywords: has-patch has-unit-tests | Focuses:
--------------------------------------+---------------------
Comment (by hellofromTonya):
Replying to [comment:23 dmsnell]:
>That being said, it sounds like the main complaint is people thinking
that skipped tests implies they should be doing more work on something
they are unfamiliar with?
From me, no this isn't the case.
My concerns are:
* Impacts to the test suite performance.
>these skips also consume resources such as adding processing time to the
test suite.
Efforts have been and are underway to make the test suite faster and use
less resources. The more non-actionable tasks its does (such as skipping
tests), the slower it runs and more the resources it uses.
* Harder for those not working on the HTML API to parse the results.
>the skipped tests messages pollute the test suite's results, making it
harder for contributors (of other tests and code beyond the HTML API) to
scan through the results to find relevant information.
While I appreciate the intent and impact to those working on the HTML API,
my concerns are on the impacts to Core's test suite and the CI jobs.
Thus, I'm curious:
What is possible to avoid skipping and not running unsupported datasets in
the tests?
>manually sift through the thousands of tests and break them up into
pieces
Hmm, this raises another concern:
What is the impact to Core's test suite runtime and resource consumption?
As I noted previously, an effort has been and is underway to improve the
test suite's runtime and performance. While this effort should not block
adding more and more code coverage, where possible, the performance of the
test suite should also be considered.
So if hundreds or thousands of tests are unsupported and will be skipped,
IMO these should not be yielded as a test dataset, i.e. at least not
without assessing the impacts to the test suite itself.
--
Ticket URL: <https://core.trac.wordpress.org/ticket/60227#comment:24>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform
More information about the wp-trac
mailing list