Allow search engines to crawl and index blog

11 years 9 months ago #206393 by KillerSurfLinks
Allow search engines to crawl and index blog was created by KillerSurfLinks
How would we set up our forum and blog to be crawled and indexed by search engines?

Is it as easy as adding an allow path to our robots.txt file? If so, which path would we use?

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/

Mike

Please Log in to join the conversation.

Moderators: beatnantkrileon
Time to create page: 0.124 seconds

Facebook Twitter LinkedIn