[wp-hackers] compression and caching

[Otto]

On Mon, Dec 28, 2009 at 2:34 PM, Micah <wp at twolanedesign.com> wrote:
> So this is the root argument against integration into Wordpress. Not
> everyone does everything the same way and most people don't need to learn
> Apache in order to use compression when plugins are created for this
> purpose. I get it but I don't agree.
>
> Compression can't be turned on by default. Bad idea.

Actually, that's the correct way to do it. There's dozens of different
possible ways to optimize a site, should every single possible one of
them be integrated into the core as well?

Simple techniques like whole-page caching or server-side compression
don't fit every site. Not just because of bad things when you try to
use them inappropriately, but because there's lots of different ways
of doing them, and people who need these sorts of things can choose
from different solutions that fit their needs best.

When you put something in core, you tend to pre-empt it from the
plugin realm. Caching and compression don't need that sort of
pre-emption, they need variety.

> While I'm about to make a huge
> assumption, I think the majority of Wordpress installations are standard,
> shared server installations.

That's a MASSIVE assumption there, and probably a wrong one at that.

Also, most people installing on shared server installations are the
ones that *don't* need compression. Compression adds a heavy amount of
CPU load, which is your most limited factor on a shared host. Caching
can be useful on shared hosting, unless you're on some form of grid
hosting in which case file-caching to a network filesystem would cause
you to become much slower than normal.

You can't assume anything. That's sort of my point. All the things
you're advocating have downsides, and even those are in cases that are
commonplace.

> the rise of Wordpress as a CMS should put light on
> creating a better end user experience out of the box, not just with a lot of
> plugins.

Neither of these creates a better end user experience in 95% of the
cases. Compression doesn't magically make things faster. Compressing
things on-the-fly is slower than not compressing them half the time.

Compression saves your bandwidth.
Caching saves your CPU time.
Both together *can* make a site speedier, but **they usually don't**.
Not all the time, and not even a majority of the time. And there are
severe compatibility issues to contend with as well.

What I'm saying is that because things can go wrong, and will go
wrong, in a not insignificant number of cases, this is and should be
plugin material.

FWIW, I'm fairly sure that WordPress.com doesn't use whole-page
caching; every one of those 20,000 page requests they get every second
is generated dynamically. They do use memcached object caching (or
similar), as I recall hearing somewhere.

> Isn't maintaining the gotchas what already happens to WP Super Cache
> support? It seems logical that since the core developers can control all
> aspects of the hows and whats about caching a dynamic system, knowing how
> brilliant you people are already, I don't think this problem gets much worse
> in the core.

Actually, it gets a thousand times worse in the core. Object caching
was removed and made pluggable for that very reason.

> Compression will always make a file smaller. Smaller file sizes means less
> bandwidth. Less bandwidth means faster page loads. This isn't blanket
> statement.

Not only is that a blanket statement, it's a series of *wrong* ones.

Compression takes CPU time. CPU has to process instructions to
compress the data. While the data is being generated, browser is
waiting for data. Data cannot be transmitted until compression is
complete, as compression is not a streaming operation in most cases.
Thus, page takes *longer* to display.

This happens in about 35% of cases, depending on server load. If you
combine full page caching into the mix, then yes, you can achieve much
by eliminating the compression itself, at the cost of making your page
static instead of a dynamic page (a cost that many people don't want
to pay).

> Plugins are third-party to the core software. Including these techniques
> into the core takes away the middle man of having to use a plugin. I'm not
> against plugins, I just think root functionality that helps end user
> experience shouldn't be dismissed as plugin material.

First, understand that plugins are NOT slower. Quite the contrary, in
fact. Plugins can specialize. The core cannot. Specialized code is
often faster than generalized code, because it can afford to exclude a
percentage of the possible population.

Also, plugins are NOT middle-men. What exactly do you think the
difference is, technically, between a plugin and core code? Because
*there isn't any*. When WordPress loads, one of the first things it
does is to load the list of plugins and include them, one by one. You
know what it does to load other pieces of the WordPress core code? It
includes them.

WordPress plugins are loaded and executed in the exact same way as
every other file in the whole thing. Half of the internal "core"
functionality of WordPress even uses actions and filters in the same
way plugins do (see default-filters.php).

> I know Wordpress is already complicated as it is. Adding tons of plugins
> further complicates this CMS and add more requests to slow it dwn.

If you put those pieces in the core, then you get the exact same
slowdowns, only now you get them even if you *don't* use that piece of
code.

> In the
> end, the CMS is a more efficient, dynamic way to publish a website and the
> end user experience can be improved immediately with core compression and
> caching functionality.

I'm sorry, but you are simply mistaken. I'd be glad to further expound
upon why you're mistaken, but I'd prefer to do it off list. People get
cranky when I go on list with this sort of thing. You can email me
directly if you like at otto at ottodestruct.com and I'll be glad to
continue this part of the conversation.

-Otto