Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Hispiforce

1. A guy who has Aids PERSON 1: HEY HE HAS AIDS PERSON 2: NO, HE IS HISPIFORCE! See hispiforce, gourav, aids, cure, no..

John Walsh

1. he is the Host of Americas Most Wanted on Fox he helps Tracks Down Murders Gangs,peodphille,and Etc and john askes you to call if you ha..

Yaynessed

1. The state of having a lot of yayness, or generally being in a very good mood. It can often result in other people becoming annoyed at th..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning