[wp-trac] [WordPress Trac] #55443: Create WebP sub-sizes and use for output

[WordPress Trac]

#55443: Create WebP sub-sizes and use for output
-------------------------------------------------+-------------------------
 Reporter:  adamsilverstein                      |       Owner:
                                                 |  adamsilverstein
     Type:  enhancement                          |      Status:  assigned
 Priority:  normal                               |   Milestone:  6.1
Component:  Media                                |     Version:  6.0
 Severity:  normal                               |  Resolution:
 Keywords:  has-unit-tests needs-dev-note        |     Focuses:
  needs-docs needs-user-docs needs-patch 2nd-    |  performance
  opinion needs-testing changes-requested        |
-------------------------------------------------+-------------------------

Comment (by adamsilverstein):

 Thanks for the feedback @azaozz! Responses and sone suggested ways forward
 below...

 First: on the specific point of the default behavior, I agree with your
 suggestion of only generating WebP format images. Also, I like the
 possibility of having a (filterable) “threshold” value above which (only)
 JPEGs are generated, for example making anything over 1024 or 2048 pixels
 in either dimension default to JPEG - similar to the way we handle
 big_image_size_threshold/-scaled images currently.

 The performance team will research this more directly to test what the
 threshold should be (and the assumption that we should have a threshold).
 Anecdotally in the field data we can see that images on desktop WordPress
 pages using WebP images are about 8% smaller, while images on mobile pages
 using WebP are around 38% smaller (based on HTTP Archive data using this
 query). One advantage of retaining some sizes is we can also use these
 JPEGs as fallback images for the edge cases where WebP still isn’t
 supported (eg. Open Graph tags).

 Second: on the infrastructure added here to support multiple mimes. This
 includes the code added to support multiple mimes directly in image
 generation, as well as changes to core media functions and on the fly
 updating on post content. We need to weigh the pros and cons of these
 changes separately.

 Pros:
 - Prepares media for supporting multiple types using the picture element
 - Enables seamless support for multiple mimes useful for plugins managing
 media.
 - Allows for maximum backwards compatibility because current data/types
 remain unchanged (when generating both types).
 - Requested by contributors: in the original development issue, only WebP
 images were generated; the multi-mime type support was added based on a
 contributor issue and subsequent group decision (worth reading for
 context).
 - Supports any use case where multiple mimes are desired by the site, for
 example generating JPEG fallback images on demand
 Cons:
 - Adds additional complexity/maintenance burden to core media (which is
 already complex)
 - Single/alternate mime type output can still achieve a high degree of
 backwards compatibility as long as we keep some versions as JPEGs.
 - If we generate multiple MIMES:
  - Consumes significantly more resources on upload (increasing the chance
 for failures)
  - Small overhead of run-time content image swapping
  - Degraded user experience (slower uploads)

 Given these points I can see two paths forward we should consider:

 1. Keep the current multi-mime infrastructure, but change the defaults so
 only WebP files are generated, possibly up to a threshold size beyond
 which only JPEGs would be generated. Most existing work would continue;
 content filtering could possibly be removed.
 2. Revert the multi-mime infrastructure and switch back to a single mime
 approach, using WebP for images up to a threshold size and adjusting the
 compatibility layer to use the JPEGs we keep.

 When making this decision, we should also consider what is coming in the
 next few years: namely widespread AVIF support (support will land shortly
 in Safari).  If we might be adding AVIF support in a few years, would we
 want the multi-mime support then, or would the “single mime” output
 support suffice?

 My own inclination is to go with option 1 because of the pros listed above
 and the community process that led to the approach (and possibly I will
 admit some attachment to all the hard work that went into developing it).
 It works well, adds flexibility to media handling and sets us up for the
 future.

 I recognize option 2 would be a much smaller change and would involve less
 risk and maintenance burden while still achieving the main goal of
 improving WebP adoption and hence lifting WordPress site performance. If
 we go with this option, we can still consider adding multi-mime support in
 the future, including the async “on demand” generation you suggested so we
 could actually improve upload success and user experience.

 Let me know what you think for the best path forward and my
 points/questions above. Did I miss any pros/cons of multi-mime support?
 I'd like to agree on an acceptable approach soon so we can land this
 before beta (Scheduled for Sept. 20 according to
 https://make.wordpress.org/core/2022/06/23/wordpress-6-1-planning-
 roundup/).

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/55443#comment:128>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform