Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Kowaltchuked

1. To steal, borrow without ever returning or scury away with someone's lighter. Can be intentional, unintentional or second nature. ..

Kishi

1. Slang for cutey or "Kish" for cute. derived from gambian word of same meaning Man, that kid is Kish. hey kishi, wanna bowl? ..

Quaker Steak And Lube

1. The Best Restuarant ever! Quakers Wings are the best 1. The Best Restuarant ever! Quakers Wings are the best..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning