Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Gweh

1. go away - carribean/jamaican See keya..

Chuntified

1. a person who has changed to be chunti That person got chuntified See beaner, wetback, chunti, chuntis, hispanic..

University Of Northern British Columbia

1. A university located in Prince George, British Columbia. Known as UNBC and the University of No Better Choice. I want to go to the Univ..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning