[wp-hackers] Robots.txt

[Robert Deaton]

I personally think this whole discussion is a moot point. Maybe we
should throw a blank index.php file into wp-content to make sure that
robots can't get a directory listing, but other than that, there isn't
any information a robot would gain from spidering, other than a
WordPress login page. If users feel the need to protect themselves
more, they can add their own robots.txt.

On 11/23/05, Roy Schestowitz <r at schestowitz.com> wrote:
> _____/ On Wed 23 Nov 2005 16:46:50 GMT, [Denis de Bernardy] wrote : \_____
>
> >> -----Original Message-----
> >> From: wp-hackers-bounces at lists.automattic.com
> >> [mailto:wp-hackers-bounces at lists.automattic.com] On Behalf Of
> >> David Munn
> >> Sent: Wednesday, November 23, 2005 5:45 PM
> >> To: wp-hackers at lists.automattic.com
> >> Subject: [wp-hackers] Robots.txt
> >>
> >>
> >> I was thinking it would be a good idea to include a simple robots.txt
> >> file within the Wordpress release to block of bot access to the
> >> wp-admin, wp-includes, and wp-content directories.
> >>
> >> opinions?
> >>
> >> David Munn
> >
> > make that a robots-sample.txt
> >
> > I'd hate to loose my robots.txt by mistake.
> >
> > D.
>
> Not everyone installs WordPress at the top level either. The implication of
> multiple robots.txt files would be user confusion. I suggest prepending to the
> top of each administration page e.g.
>
> <META HTTP-EQUIV="CACHE-CONTROL" CONTENT="NO-CACHE">
> <META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW">
>
> Roy
>
> --
> Roy S. Schestowitz
> http://Schestowitz.com  |    SuSE Linux     |     PGP-Key: 0x74572E8E
>   4:15am  up 21 days  0:09,  4 users,  load average: 0.18, 0.33, 0.26
>
> _______________________________________________
> wp-hackers mailing list
> wp-hackers at lists.automattic.com
> http://lists.automattic.com/mailman/listinfo/wp-hackers
>


--
--Robert Deaton
http://somethingunpredictable.com