Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Lake Town

1. lake town is slang for salt lake city.. h-town iz for houston lake town is for salt lake city See lake, salt lake..

Farmers Daughter

1. Slang for a quarter of an ounce henry today? no, better make it a farmers daughter See deadwood..

Kaboshed

1. The act of being extremely drunk. Brad was kaboshed at the bar last night. See smoked, lit, bombed, cooked, tanked..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning