Robots.txt

What is Robots.txt?


1.

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM


35

Random Words:

1. A guy who has Aids PERSON 1: HEY HE HAS AIDS PERSON 2: NO, HE IS HISPIFORCE! See hispiforce, gourav, aids, cure, no..
1. he is the Host of Americas Most Wanted on Fox he helps Tracks Down Murders Gangs,peodphille,and Etc and john askes you to call if you ha..
1. The state of having a lot of yayness, or generally being in a very good mood. It can often result in other people becoming annoyed at th..