Robots.txt

What is Robots.txt?

1.

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM
35

Random Words:

Liarsexual
1. Transexual. One who lies about their physical sex and sexual organs. Butch lesbian. Drag Queen. Pre-op M-to-F. F-to-M with a whole lot o..
Read more
Zamboni Driver
1. caucasion female enjoying canadians redefined for pleasure and dating. check out the zamboni driver with the canadian. See conductor, ..
Read more
Nouri Al-maliki
1. The current prime minister of Iraq; elected in 2006. Constitutional mandate expires in 2010. During his first year and a half in power ..
Read more

Best StartPage for your browser.

All you need for a faster workflow in one place.



Get it for Free