[wp-trac] [WordPress Trac] #65376: Build/Test Tools: Statically verify that hook documentation and arguments stay in sync

[WordPress Trac]

#65376: Build/Test Tools: Statically verify that hook documentation and arguments
stay in sync
------------------------------+-------------------------
 Reporter:  westonruter       |      Owner:  westonruter
     Type:  defect (bug)      |     Status:  assigned
 Priority:  normal            |  Milestone:  7.1
Component:  Build/Test Tools  |    Version:
 Severity:  normal            |   Keywords:
  Focuses:  docs              |
------------------------------+-------------------------
 WordPress hooks rely on conventions that are currently only enforced by
 human review, which means they drift out of sync over time.

 == The problem ==

 Every `apply_filters()` and `do_action()` call is expected to be preceded
 by a docblock documenting the hook, or by a reference comment pointing at
 the canonical docblock when the same hook fires in more than one place,
 for example:

 {{{
 /** This filter is documented in <file> */
 }}}

 Nothing verifies any of this automatically:

  * '''Reference comments are not checked.''' A `This filter is documented
 in <file>` comment can point at a file that no longer exists, or at a file
 that does not actually document that hook. These references have to be
 manually traced to confirm they are still accurate, and they silently rot
 when hooks are moved or renamed.
  * '''Argument counts are not checked.''' A hook's docblock lists the
 arguments it passes, and WordPress forwards those arguments straight to
 each callback. When a call site passes fewer arguments than documented, a
 callback registered for the documented count fails with an
 `ArgumentCountError` (or a `TypeError` for a typed parameter) at runtime.
 When it passes more, the extra argument is silently dropped and the
 documentation is misleading. Today the only way to catch either is to
 manually compare every call site against the docblock.
  * '''Undocumented hooks slip through.''' A new
 `apply_filters()`/`do_action()` with no preceding docblock or reference is
 not flagged, so gaps accumulate.
  * '''Documentation itself is unverified.''' Malformed `@param` tags (e.g.
 missing the variable name) and stale references are invisible until
 someone reads the file closely.

 I realized the need for this when [https://github.com/WordPress/wordpress-
 develop/pull/12007#discussion_r3325693119 reviewing] a PR for #65367, as
 the file reference was incorrect but I had to manually check it for
 accuracy.

 In regards to argument counts no being checked, consider the following
 code:

 {{{#!php
 <?php
 add_filter(
         'https_local_ssl_verify',
         static function ( bool $ssl_verify, string $url ): bool {
                 if ( str_contains( wp_parse_url( $url, PHP_URL_HOST ),
 'local' ) ) {
                         $ssl_verify = false;
                 }
                 return $ssl_verify;
         },
         10,
         2
 );
 }}}

 The filter callback is expecting two parameters, `$ssl_verify` and `$url`,
 according to the filter
 [https://developer.wordpress.org/reference/hooks/https_local_ssl_verify/
 documentation]:

 {{{
 /**
  * Filters whether SSL should be verified for local HTTP API requests.
  *
  * @since 2.8.0
  * @since 5.1.0 The `$url` parameter was added.
  *
  * @param bool|string $ssl_verify Boolean to control whether to verify the
 SSL connection
  *                                or path to an SSL certificate.
  * @param string      $url        The request URL.
  */
 }}}


 Nevertheless, the above filter code currently causes a '''fatal error'''
 when loading Site Health:

 > Uncaught Error: Too few arguments to function {closure}(), 1 passed in
 /var/www/src/wp-includes/class-wp-hook.php on line 344 and exactly 2
 expected

 This is because the filter is [https://github.com/WordPress/wordpress-
 develop/blob/f19efd0e5b3cb774a330f16d2eb3486fbd743624/src/wp-
 admin/includes/class-wp-site-health.php#L2215-L2216 applied] in
 `WP_Site_Health::get_test_rest_availability()` //without// that second
 parameter:

 {{{#!php
 <?php
 /** This filter is documented in wp-includes/class-wp-http-streams.php */
 $sslverify = apply_filters( 'https_local_ssl_verify', false );
 }}}

 Since the `$url` parameter was added in 5.1.0, apparently this argument
 was not also added to all other instances of the `https_local_ssl_verify`
 filter being applied, or actually in this case the filter was added in
 5.3.0 via r46231 and this parameter was missed.

 Because these checks are manual, mistakes happen readily: there are hooks
 fired with the wrong number of arguments across taxonomy, multisite, XML-
 RPC, plugin, menu, and REST API code, references pointing at an obsolete
 file, malformed `@param` tags, and bundled-theme calls that did not pass
 the documented arguments.

 Separately, the value returned from `apply_filters()` is typed as `mixed`,
 even though the hook's docblock already declares the type of the value
 being filtered. This discards type information that is sitting right there
 in the documentation and weakens static analysis of any code that consumes
 a filtered value.

 == Proposed solution ==

 Teach PHPStan to read the existing hook docblocks so these conventions are
 verified automatically rather than by hand:

  * Validate that every hook invocation is documented, that "documented
 elsewhere" references resolve to a real docblock for that hook, and that
 each call passes the documented number of arguments.
  * Use the documented `@param` type as the return type of
 `apply_filters()`.

 Then fix the issues this surfaces in core.

 As with how #64898 implemented support in a bundled PHPStan extension for
 the `@global` tag in WordPress conventions, I suggest the above be
 implemented as bundled PHPStan extensions to start with. The great work of
 @szepeviktor in [https://github.com/szepeviktor/phpstan-wordpress phpstan-
 wordpress] can serve as a starting point. However, once landed in core,
 they should then be submitted upstream for adding to `szepeviktor/phpstan-
 wordpress` so they can more easily be reused by both core and Gutenberg
 (once [https://github.com/WordPress/gutenberg/issues/66598
 gutenberg#66598] is completed).

 This is part of #64898 and #64896.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/65376>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform