Robots.txt

What is Robots.txt?


1.

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM


35

Random Words:

1. A slang word that means stuck between dimentions. Alex got was zipsy when he ate potato. See sphinkter, pooz..
1. The number coming after 19 in hexadecimal. 9, A, B, C, D, E, F, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1A See hexadecimal, numbers, d..
1. an abbreviation (abbrevs, if you will) for ipod headphones "shit, i forgot my ipes headies at home"..