[wp-hackers] Rewriting search URL

[Mark Jaquith]

Sebastian Herp wrote:

> Matthew Thomas wrote:
>
>> Peak Discharge wrote:
>>
>>>
>>> That changes my url into the form of 
>>> http://www.unicorns.com/?s=string, which is a progress. The aim, 
>>> however is to change all search urls to 
>>> http://www.unicorns.com/search/string
>>> ...
>>
>>
>>
>> Sorry to crash the coding party here, but could I just point out that 
>> this is rather a bad idea?
>>
>> Because an URL like this doesn't have any "?"s in it, Google (and 
>> other search engines) won't know it's a page of search results, so 
>> whenever someone happens to link to it, search engines will index it 
>> *in addition to* the actual article pages. The end result will be 
>> that search engines are cluttered with result pages from your own 
>> search engine (indeed, I've seen sites trying to game Google using 
>> this technique). Worse, the results will often be irrelevant because 
>> they'll be plucking keywords from the summaries of several distinct 
>> Weblog entries.
>>
>> If you want cruft-free search result URLs, I suggest aiming for 
>> something like <http://example.com/search?cat+pictures>. Same number 
>> of characters, but the ? tells indexers (and geeky humans, who should 
>> not be neglected) that the page quite probably isn't static writing.
>>
> Good point, but I have linked a search result in one of my blog posts 
> and I am having visitors from Google for exactly that search. If there 
> is a link to a search, google (and others) are likely to include it in 
> their index (unless you add some special meta tags or robots.txt). If 
> there are is no link, how should google ever know what to index? It's 
> not like the googlebot sits in front of my searchform and tries 
> different phrases ;-)

In robots.txt:


User-agent: *
Disallow: /search/