Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Boom Boom Dollar

1. one of the original licensed songs to ever appear in Dance Dance Revolution Boom Boom Dollar is a classic...

Salua

1. salua are people that have HUGE ASSES usually they're hot and tanned and the only reason people hate a salua is because they&apos..

W0rd N Shit

1. when someone is agreeing with what someone else is saying. Also could be used in the beginning of a sentence if the person talking is e..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning