Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Bwbf

1. Acronym for Bitches Wanna Be Fucked. Describing all of those skanks who put out, giving guys the impression that all bitches truly do w..

Quantum Ninja

1. 1) a person with extreme ninja skillz 2) quantumninja.com - cool website dude that guy that just kicked my ass was a quantum ninja 1...

Bumpass

1. An individual, generally female, who exhibits a well defined and prominent glutinous maximus. An essential element of a bumpass is the a..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning