Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Cas

1. Abreviation that stands for confidence, appearance, and swagger. The result of having CAS is an emitted aura from one's body like w..

W.a.w.a.

1. Acronym meaning White Ass Whack Assholes. Refers to cocky ass jock, frat, white boys and their pasty, orange-tanned, dumb, blonde ass g..

Irish Matt

1. an irish matt is when performing oral intercourse between two lesbians they both squirt their love juice on each others faces and drink ..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning