Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Leazer

1. monster: described as a fat albino sack of shit, who is self sentered and is going to eat your babies. Some day IT will come to your hou..

Internet Vixen

1. An Internet Celebrity that is seducing. Tom: Joe Hollywood is an Internet Celeb. Sarah: More like an Internet Vixen, baby! See music,..

Prack

1. To have the perfect "rack" as in breasts. Preferably double D, but not like yo grandma's double D's. Also preferably..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning