Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Infelicitous

1. not felicitous,not appropriate or well-timed,awkward the infelicitous typesetting was due to illegible copy" See unfortunate, ina..

All Yalls

1. yall + 3 or plus two if one of ems fat all yalls get in the car See rubberduck..

Braemartian

1. A Scotsman who is equipped with a CABER so awesomely large that it appears to be from another planet. At this year's Braemar Highl..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning