Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Intellectual Orgasm

1. to enjoy a high- level or intelligent conversation to the maximum. Having heard their story ,I felt an intellectual orgasm. See enjo..

Quench Buddy

1. A person who always bring beverages to the potluck, such as Mountain Dew and Dr. Pepper, and is considerate enough to provide them in ca..

Yo, You Good?

1. wtf is wrong with you? john just spilled beer on the new rug... john! yo, you good? See aight, ok, yo, wtf..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning