Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Mat Fag

1. A mat fag is a homosexual amateur wrestler. Most mat fags only wrestle because of the sexual pleasure they derive from participating in ..

Bimbo Baker

1. This is when there are cushions on the out on top of the bow of the boat, so semi-attractive girls, often ones in skimpy bathing suits, ..

Swillie

1. an english banger, the official dish of a royal goat-molester. "Can I put my swillie in your bun? Relish anyone?"..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning