Robots.txt

What is Robots.txt?

File that stops search engines from crawling a specified page or directory.

Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

See KFM

Sfy

1. "Sorry For You" = SFY used as a brief acronym to show you feel for someone in chat conversations. Gernally said in a sarcasti..

Ovarieasy

1. when a girl is easily impregnated. either people dont know how to wrap up, or sheila is ovarieasy See ovaries, pregnant, sex, slut, sp..

Violently

1. Often used to describe something intense, meaning in a violent way maybe involving some shaking/twitching. Person 1: "Wow, look at..

All you need for a faster workflow in one place.

Robots.txt Edit Meaning