Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Bingle Snap

1. noun. a small flowering shrub that is said to consume human flesh when anyone gets near them. We grew a whole garden of bingle snaps, b..

Erotisized

1. To have something erotic occur to you. The cat erotisized your face. (teabagged) See erotic, teabag, naughty, hump..

Howell Avenue

1. One of the streets found in Clifton, which I sometimes frequent. I found great parking on Howell Avenue!..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning