[wp-hackers] GSoC Proposal: Integrate WP-cache / WP Super Cache into WordPress

Sun Mar 2 07:44:00 GMT 2008

Very curious about that front page...

Having several WordPress blogs that are under regular Digg/Reddit  
stress, i've found it hard to keep those blogs alive...

If such a project would start, feel free to get me onas a tester. I've  
got quite some experience with load testing.

- Joost

Sent from my iPod

On Mar 2, 2008, at 8:06, "Computer Guru" <computerguru at neosmart.net>  
wrote:

> I've worked extensively with WordPress caching projects (both HTML and
> object caches, having rewritten WP-Cache for IIS, helped Donncha with
> a bunch of WP-SuperCache stuff, and written caching engines for XCache
> and eAccelerator) and I've come to one really, really scary
> conclusion: all they do is make up for the original WP bloat.
>
> And, believe me, I don't say this lightly. I was getting real fed-up
> with WordPress dying under joint Slashdot-Digg traffic no matter what
> caching solution I used on a dedicated host so I decided to rewrite
> the main page and the single-post pages in plain PHP from scratch. I
> did - plain PHP & HTML with some simple EZSQL functions, hard-coded
> themes, etc. and the difference was unbelievable.
>
> See for yourself:
> http://neosmart.net/blog/
> vs
> http://neosmart.net/blog/page/2/
> (page load times at the bottom, current iteration *does* make use of
> *object* caching, no HTML cache)
>
> Whereas WP would take 5+ seconds under a *full* load to create a
> single page (and far, far too many queries), the rewritten frontend
> never went above .01 seconds/page on average at the peak of the
> digg-slashdotting effects.
>
> Bottom-line: Caching, in all its forms, is a way to get that last bit
> of performance boost to make your app a real killer. It is NOT a way
> to make up for badly-performing code in the first place.
>
> I understand the difficulties of making flexible-yet-well-performing
> code. I know the tradeoffs between hard-coding stuff and having good
> page-load-times. But at the end of the day, /caching is NOT
> WordPress's biggest problem/, rather loading too many files at once,
> using way more nested loops than it should, and not skipping certain
> sections of the code when it can are what's responsible for WP's
> "legendary" fear of the /. effect.
>
> A well-written package doesn't need caching of any sort or form, just
> a simple opcode cacher in the backend should more than suffice. The
> replacement frontend for WP (which I'm considering releasing) was
> generating pages in 0.00xx seconds, w/ object caching it does it in
> 0.000x seconds instead (warm hits, that is). I had attempted to
> object-cache everything I could see in a stock WP install, but that
> didn't give me anywhere near the performance boost I was looking for.
>
> I'm willing to mentor any performance-related projects for WP.
>
> On 3/1/08, Andy Skelton <skeltoac at gmail.com> wrote:
>> On Fri, Feb 29, 2008 at 12:17 AM, Ronald Heft  
>> <ron at cavemonkey50.com> wrote:
>>> excellent opportunity to start fresh with a new caching system. If  
>>> anyone
>>> has any examples of a PHP-based caching system in they feel is  
>>> better than
>>> WP Super Cache, I would love to see it.
>>
>>
>> I've done a lot of work on the caching systems that WordPress.com
>> uses, severely hacking on Ryan's memcached client, and I wrote the
>> HTML cache that helps burst-traffic event blogs like live.gizmodo.com
>> stay up when most others fail.
>>
>> I don't know of anything better than Donncha's Super Cache for what  
>> it
>> does. My HTML cache (I named it supercache before Donncha released  
>> his
>> Super Cache, but I never released mine so he won the name recognition
>> in the end) is based on memcached and that makes it a poor fit for  
>> the
>> mainstream right now. That's not because memcached is hard to set up
>> (it's not) or hard to code for (we have working clients) but shared
>> hosting setups typically don't allow it.
>>
>> Some form of worthwhile persistent caching should work in the default
>> install with the minimum requirements. Obviously we aren't there yet.
>> If we keep the requirements as they are, we will have to put the  
>> cache
>> in the database. Caching in the database would only be worthwhile if
>> we cached very high-level objects, the highest level being fully
>> rendered pages. Obviously this would not be as good as an APC object
>> cache, but for the minimum install it could do wonders.
>>
>> What I learned from my experiences with WordPress and caching is that
>> there are more variables than you can hope to account for. (You can
>> take the WordPress out of the dynamic but you can't take the dynamic
>> out of WordPress?) You can get a lot of mileage from caching at any
>> level (queries, values, objects, rendered HTML elements, entire  
>> pages)
>> but the higher up that stack you go, the harder it gets because there
>> are so many factors contributing to the rendered page.
>>
>> Back to my supercache: it's painfully simple and extremely effective
>> at serving a page to new visitors. It allows pages to be freshly
>> rendered for cookie holders or whatever criteria you care to  
>> configure
>> for identifying visitors who require it. The whole thing is about 200
>> lines of PHP including comments. Using a similar test for cache
>> eligibility, a db-backed HTML cache could extend the operational
>> availability of sites that experience surges of new visitors to  
>> single
>> posts (a typical Digg scenario). This would have to be significantly
>> more complex than my supercache because we would have to code even  
>> the
>> basic mechanism of expiration and cleanup into the cache. However, if
>> we are only caching entire pages, this can be pretty easy.
>>
>> One thing that kills an HTML cache is too-dynamic content. Random
>> output (e.g. random-ordered blogroll), time-dependent elements (e.g.
>> clocks), and HTTP header-dependent elements (e.g. seach term
>> highlighting [Referer] or Firefox buttons [User-Agent]) all become
>> frozen in the cache and should not be used on blogs that serve
>> pre-generated pages unless they are generated client-side with JS,
>> Flash, etc., or the cache is set up to vary accordingly.
>>
>> Wrapping up, I think there should always be attention devoted to
>> improving WordPress resilience on the minimum setup and I would like
>> to see this as a GSoC project.
>>
>> Cheers,
>>
>> Andy
>>
>> _______________________________________________
>> wp-hackers mailing list
>> wp-hackers at lists.automattic.com
>> http://lists.automattic.com/mailman/listinfo/wp-hackers
>>
>
>
> -- 
> Computer Guru
> Director,
> NeoSmart Technologies
> http://neosmart.net/blog/
> _______________________________________________
> wp-hackers mailing list
> wp-hackers at lists.automattic.com
> http://lists.automattic.com/mailman/listinfo/wp-hackers