Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Ertw

1. Abbrev. for "Engineers Rule The World" 2. Acronym for "Engineers Rarely Touch Women" If ERTW, how do they reprodu..

Assumptious

1. adjective to frequently assume things Katie thinks whenever she sees a guy stumbling around he must be drunk. She's very assumpti..

Yoda-talk

1. Changing the order of words in a sentence as the Jedi Yoda from Star Wars would. It makes the sentences a drunk person speaks funnier a..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning