Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Jibberish

1. A real language that girls use to tell each other things so boys won't know what they're saying. dithagont yithagou withagish..

Ohmigoshies!

1. A silly slurred version of "Oh My Gosh!" You can probably find a hyperactive immiture girl saying this. Ohmigoshies, It'..

In The Bacon

1. to be closely spaced together, or tightly packed as bacon is in a frying pan. "All 10 of us drove Route 66 together in a Civic...w..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning