Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Zomg'd

1. 1. Verb, completely beaten or owned,usually seen with an exclamation of ZOMG! Carl got zomg'd by Lucas..

Lorton

1. 1. A town in Virginia. 2. A state of mind one enters after a large amount of drinking. To enter said state one usually must consume ..

J-mo

1. Jim Morrison, rock legend, of the band The Doors. "J-Mo wrote that song when he was sober." "... Okay, so maybe he didn..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning