Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Jew Rat

1. A Jew Rat or Deerty Jew Rat is the term given to jewish people who are frequently scouring the city streets and sewers for any quarters ..

Getting G'ed

1. being screwed or haveing sex. your getting g'ed tonight. See getting g'ed, getting laid, sex 2. being screwed or haveing s..

Shmingy

1. pronounced shming ee or SHminG-E like one would pronounce 'sing-E' with a SHM instead of just s An extraordinarily odd per..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning