Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Chee Siong

1. A kind of curse, smiliar to 'damn'. You're such a chee siong! See damn, chee, curse, tcb..

Break Nut

1. The act of male ejaculation. I'd like to break nut across Britney Spears' jaw...

Twas

1. 1)a possessive form of the airline abbreviation 2)The plural of "twat" 3)A toast made a bit later in the evening. 1)You&ap..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning