Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Virord

1. A word spread by viral marketing. Derived from viral word. Virord is a viral word, virord is virord. have you heard that great new viro..

Krishanu

1. A person with a small penis. person with a penis the size of this dot...

Yacht

1. A furnished yet elegant ship/vessel, used for private parties or pleasure. A recreational boat. Rich people like Yachts. See boat, sh..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning