Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Sactor

1. As said by Devon Soltendieck, a combination of the words singer and actor, most of the time people have talent in one area and think the..

Live-blogging

1. Live-blogging is what one does when covering an event and posting to a blog in real time as a "citizen journalist." A blog is..

Kussy

1. Derived from the words Hussy and Cunt. A very stupid and/or idiotic human being. Marsha Brady is a kussy!..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning