Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Nubbah

1. 1.a awp,deagle, And ak47 whore in counter strike 2. A person who sucks and use hacks to save their nerd ass in counter strike 1."..

Hip-hopreneur

1. hip-hop entrepreneur = hip-hopreneur The following is a short list naming those that are classified as a hip-hopreneur: Russell Simmon..

Quaj

1. when a girl gets cum on her mouth and licks it off also can mean penis his quaj gave me quaj 1. when a girl gets cum on her mouth and..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning