[wp-trac] [WordPress Trac] #64959: Add native llms.txt and llms-full.txt support (virtual files, similar to robots.txt)

[WordPress Trac]

#64959: Add native llms.txt and llms-full.txt support (virtual files, similar to
robots.txt)
-----------------------------------+-------------------------------------
 Reporter:  serhiipylypenko        |      Owner:  (none)
     Type:  enhancement            |     Status:  new
 Priority:  normal                 |  Milestone:  Awaiting Review
Component:  Rewrite Rules          |    Version:
 Severity:  normal                 |   Keywords:  has-patch needs-testing
  Focuses:  template, performance  |
-----------------------------------+-------------------------------------
 == Problem

 The [https://llmstxt.org/ llms.txt standard] defines two files that help
 Large Language Models understand website content:

 * `llms.txt` — a short summary with links and descriptions (analogous to a
 sitemap for AI)
 * `llms-full.txt` — a comprehensive version including full page content

 Major WordPress SEO plugins (AIOSEO, Yoast, Rank Math) have already
 shipped llms.txt support. However, all current implementations either
 generate a physical file on disk or rely on PHP rewrite rules that assume
 the web server and PHP share the same filesystem.

 In containerized deployments (Docker with Nginx + PHP-FPM in separate
 containers) — which are now a mainstream hosting pattern — this fails
 silently. The plugin generates the file inside the PHP container, but the
 web server cannot see it and returns 404.

 WordPress Core already solved this exact architectural problem for
 `robots.txt` by serving it as a virtual file through `do_robots()` with
 rewrite rules pointing to `index.php`. The proposed patch applies the same
 proven pattern to `llms.txt` and `llms-full.txt`.

 == Proposed Solution

 Add native support for both files following the established `robots.txt`
 architecture:

 1. Register `llmstxt` and `llmstxt_full` as public query vars
 2. Add rewrite rules: `llms.txt` → `index.php?llmstxt=1`, `llms-full.txt`
 → `index.php?llmstxt_full=1`
 3. Add `is_llmstxt()` and `is_llmstxt_full()` conditional tags
 4. Add `do_llmstxt()` and `do_llmstxt_full()` functions
 5. Core generates minimal default content (site name, description,
 published pages)
 6. Cache via transients (auto-uses object cache when Redis/Memcached
 available)
 7. Separate cache durations: 12h for llms.txt, 24h for llms-full.txt
 (filterable)
 8. Auto-invalidate on `save_post`, `deleted_post`, `switch_theme`
 9. Skip autosaves and revisions during invalidation

 == Plugin Extensibility

 The patch provides the same hook-based API pattern as `robots.txt`:

 * `do_llmstxt` / `do_llmstxt_full` — actions fired before output
 * `llmstxt_output` / `llmstxt_full_output` — filters for full output
 control
 * `llmstxt_post_types` — filter to add/remove post types (default: pages
 only)
 * `llmstxt_cache_duration` / `llmstxt_full_cache_duration` — cache TTL
 filters

 This allows SEO plugins to extend or completely replace the default
 content using standard WordPress filter API, instead of filesystem-
 dependent approaches.

 == Files Modified

 * `wp-includes/class-wp-query.php` — add `is_llmstxt` / `is_llmstxt_full`
 properties
 * `wp-includes/class-wp.php` — register public query vars
 * `wp-includes/class-wp-rewrite.php` — add non-WP rewrite rules
 * `wp-includes/functions.php` — `do_llmstxt()`, `do_llmstxt_full()`,
 `_generate_llmstxt()`, `_clear_llmstxt_cache()`
 * `wp-includes/template-loader.php` — route requests to handler functions
 * `wp-includes/query.php` — `is_llmstxt()` and `is_llmstxt_full()`
 conditional tags
 * `tests/phpunit/tests/rewrite/llmstxt.php` — unit tests (8 tests)

 == Reference

 * llms.txt specification: https://llmstxt.org/
 * Detailed write-up on the Docker/Nginx problem: https://dev.to/pylypenko
 /serving-llmstxt-from-a-dockerized-wordpress-nginx-setup-2ka2

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/64959>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform