Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Zetlb

1. "Zas, en toda la boca!", spanish for "Pow, right in the kisser!" - ZETLB! - Lol! What a fail, amigo! See pow, kis..

Unshag

1. this term comes from the word "shagin wagon" see "shagin wagon" this word just means that the car saw no action... ..

Apologuise

1. An apology given when you don't feel you did anything wrong so it will end an argument. She expected an apology, so I apologuised ..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning