Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Over-chickening

1. over producing of chickens we never hear news reports dealing with over-chickening. See over-chickening..

[draw Her]

1. This is someone that you find attractive and would like to sleep with. Came from the phrase "I'd draw her", made popular..

Mike Judge

1. The mastermind that created Beavis and Butthead,King of the Hill, and Office Space.Not only is he the coolest person alive but he starte..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning