Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Kelot

1. Tagalog slang for male Kilala mo pa yung bagong kelot na yun? Do you know that new KELOT? See kelot, boylet, male, man, guy..

Throat Oyster

1. Chunk of phlegm in you throat (the thick viscous substance secreted by the mucous membranes of the respiratory passages, esp. when produ..

Zut Alors

1. pronounced zoot-uh-lore int. shucks / shoot / holy cow / dang Zut alors! Someone has sat on my bagel! See zut alors..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning