Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Feed Her Pony

1. Used to indicate hand manipulation of a broad's mingue. Accompanied by a hand motion at the approximate height of a woman's v..

Family & Friends Discount

1. a discount you typically give your family and or friends when they ring up thru your register. most people ask for this discount even if..

90 Spot

1. The term used to describe whilst one is in public, usually at night time under the influence of alcohol, in desperate need of relieving ..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning