Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Hung Rat

1. The act of forcefully pinching one's cock and balls between the legs, then displaying the entire mess posteriorly to unsuspecting o..

Fiki

1. A Legend among stories, it is said that one fiki is born every 2,000 years. The description of a Fiki is someone who will not take no f..

Fag Cannon

1. An unbalanced weapon in first person 'shoot-em-ups'. Typically used by noobsand constalty complained about by other players. ..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning