Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Pornconsternation

1. The futile and all-consuming fixation by anti-porn people with trying to rail against and stamp out in society a pastime that has persis..

Alberkerque Snoodle

1. un like in a snoodle where 2 men rub their genatles together 2 women do it instead between 2 homosexual women See Jordan..

Sharmone

1. Michael Jacksons word. Has no meaning. He just says it now and then Sharmone! chek chek da koe! See Hugo 2. ummm Sharmone mutha fuc..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning