Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Captain Pussy Fucker

1. Legendary llama of the Half-Life modification "Team Fortress Classic" well-known for his binds i.e. "take that ya son of ..

Yeawesome

1. Stating that someone who isn't cool actually is, in order to be sarcastically bashing to them. Man, that idiot sure is freaking ye..

Flaunt

1. to show off kriselda is a flaunt See anonymous 2. To make a public and usually ostentatious show of: brandish, display, disport, exh..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning