Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Crack Pairing

1. A romantic pairing (whether it be from a video game, movie, book, etc) that is completely implausible and/or bizarre, but people support..

Samina

1. A grocery cashier/college student by day... but by night, the best animeartist in the history of the entire world! She's pretty, t..

Groping

1. General feeling up on people. Usually someone feeling someone's legs (or otherwise) under a table top or while making out. Yo, man..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning