[wp-testers] Wordpress scaling problems

[matthijs]

Recently I discovered that the current way wordpress handles permalinks is
not scalable. All rewrite_rules are at the moment held in a single database
field in the wp_options table. If you have a few dozens pages and posts, you
have maybe a few hundred rewrite_rules in it and all is well. But as soon as
you start to have a few hundred pages and attachments, the amount of
rewrite_rules explodes as well as the field size. This also depends on the
permalinks settings. On one of my sites I can't even open the database field
to take a look because my browser and text editor crash because of its
size.
At some point the rewrite_rules don't fit anymore and wordpress needs to run
thousands of queries for each page load (because wp is rebuilding the rules
each time but can't insert them in the db).

See also trac 8958
http://trac.wordpress.org/ticket/8958

I was wondering:
What are the ideas about how this issue can be solved? Are there plans to
restructure the way permalinks and rewrite rules are kept in the database?
Or on how to deal with uploads/attachments?

Wouldn't it be better to put the rules in their own db table?