Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

1. Quarter Pound of weed. Johnny the highschool dropout went back to his old school to sell QP's. See roper 2. 1. Quarter Pound of..

Fricklefrackle

1. To be very very very very funny Ha ha ha haha so funny you be a frickle frackle...

Yagometer

1. A *device/methodological approach to measuring the ones level of Yag. Yag is measured scientifically on a scale of 1-10 Elton units or s..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning