Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Yerd

1. Combination of the words Yah and Heard. In response to a question or statement. Yerd me brah? Yerd what I'm talking about? "..

Kaoklo

1. see Doggy Style. Pronounced cow clow "I want to Kaoklo that bitch." See doggy style, sex positions, mounting, cowgirl, kao..

Quantum Slut

1. A woman who detaches her head from her body so the body can commit promiscuous acts whilst the head can attend major corporate functions..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning