[wp-trac] [WordPress Trac] #57625: WP_Query cache memory leak

[WordPress Trac]

#57625: WP_Query cache memory leak
--------------------------+------------------------------
 Reporter:  owi           |       Owner:  (none)
     Type:  defect (bug)  |      Status:  new
 Priority:  normal        |   Milestone:  Awaiting Review
Component:  Cache API     |     Version:
 Severity:  normal        |  Resolution:
 Keywords:                |     Focuses:  performance
--------------------------+------------------------------
Description changed by peterwilsoncc:

Old description:

> Hey,
>
> I am not sure whether that qualifies as a bug or should we consider that
> as a possible configuration enhancement.
>
> When the queries are cached the generated cache keys are suffixed with
> the
> `wp_cache_get_last_changed` for the `posts` group (if the post contains
> tax query as well then additionally it takes `last_changed` cache key
> from the `terms`)
>
> Now this creates a problem with most of persistent cache storages which
> store keys with an infinite TTL (because why not?).
> As soon as `last_changed` key changes its value it immediately
> invalidates all the previous keys but the obsolete keys obviously remain
> in the storage forever.
>
> That may lead to memory problems as the number of keys grows over time
> (quite quickly on high traffic sites) and only way to avoid OOM issues is
> to set the expiry of the wp_query cache keys to some arbitrary number.
>
> The problem is easily reproducible with a bare WordPress 6.1 install and
> persistent object storage like Redis.
> 1. Start with empty cache storage and fill it up a bit by refreshing few
> pages through the admin or front site
> 2. You should notice that multiple refreshes of the same URL do not
> increase the amount of query cache keys
> 3. Open the new post screen (its enough to just refresh it) or do any
> action against posts.
> 4. Refresh previously open post/screen and just refresh it
> 5. Notice how new keys are just popping up in the DB as `last_changed`
> was bumped up thus invalidating previous keys.
>
> There is even shorter way to reproduce it.
> 1. Bare WP 6.1 install with object cache
> 2. Flush cache
> 3. Navigate to `new post` screen and just refresh it few times (no need
> to create post at all)
> 4. Each refresh creates new set of cache keys for queries like revisions
> etc. as every refresh touches that `last_changed` timestamp.
>

> I hope that my explanations are clear, if not then I am more than happy
> to bring more clarification.

New description:

 Hey,

 I am not sure whether that qualifies as a bug or should we consider that
 as a possible configuration enhancement.

 When the queries are cached the generated cache keys are suffixed with the
 `wp_cache_get_last_changed` for the `posts` group (if the post contains
 tax query as well then additionally it takes `last_changed` cache key from
 the `terms`)

 Now this creates a problem with most of persistent cache storages which
 store keys with an infinite TTL (because why not?).
 As soon as `last_changed` key changes its value it immediately invalidates
 all the previous keys but the obsolete keys obviously remain in the
 storage forever.

 That may lead to memory problems as the number of keys grows over time
 (quite quickly on high traffic sites) and only way to avoid OOM issues is
 to set the expiry of the wp_query cache keys to some arbitrary number.

 The problem is easily reproducible with a bare WordPress 6.1 install and
 persistent object storage like Redis.
 1. Start with empty cache storage and fill it up a bit by refreshing few
 pages through the admin or front site
 2. You should notice that multiple refreshes of the same URL do not
 increase the amount of query cache keys
 3. Open the new post screen (its enough to just refresh it) or do any
 action against posts.
 4. Navigate back to previously open post/screen and just refresh it
 5. Notice how new keys are just popping up in the DB as `last_changed` was
 bumped up thus invalidating previous keys.

 There is even shorter way to reproduce it.
 1. Bare WP 6.1 install with object cache
 2. Flush cache
 3. Navigate to `new post` screen and just refresh it few times (no need to
 create post at all)
 4. Each refresh creates new set of cache keys for queries like revisions
 etc. as every refresh touches that `last_changed` timestamp.


 I hope that my explanations are clear, if not then I am more than happy to
 bring more clarification.


 ---

 Edited per [comment:2 comment 2] below -- peterwilsoncc

--

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/57625#comment:4>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform