Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Fuck Rant

1. A venting session involving the word 'fuck', and variations of it. The fuck rant is designed to release anger. Jim: I fucking..

Quibblet

1. A poll, quiz, survey or personality test created on Quibblo. Joe: I just created a hilarious quibblet on Quibblo. Bob: What kind of qu..

Myspace Angle

1. When the camera is placed above your head at an angle to take a myspace picture. This is usually done by the horribly mangled, ugly, a..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning