Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Pink Cheeto

1. When you shove a cheeto up a girls vagina so she moans then make her eat it, and then she pees out cheesey goodness. dude, i gave this ..

Francisca

1. the hottest girl in the world. usually has gorgeous brown hair and captivating dark eyes. probably the best looking girl you will ever s..

Cocktard

1. A girl that has no idea what to do with a dick suzie's such a cocktard, she was biting me all night See Jimbo 2. adj. a word t..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning