Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Phelpster

1. A person who smokes pot but still exceptionally excels in assorted aspects of life. "Dude don't smoke pot, it will make you l..

Pouchy

1. The pocket on the front of a hooded sweatshirt, similar to that of a kangaroo pouch; widely used in the Pacific NW, origins unknown Dud..

1. an expression to denote happyness kane: i like to program java =Funkii: stop being so sad, use = more =] 2. happy. Showing pleasure....

All you need for a faster workflow in one place.

Robots.txt Edit Meaning