Viewing robot txt

robot txt



User: cecile 12 years ago
Hi,
Is EverWeb generating a "robot txt" when you publish your website. can I have more information about this text file ?
Thanks
Cécile
User: Paul-RAGESW 12 years ago
No, you don't need a robots.txt file unless you want to BLOCK search engines from seeing your site. Even then you can do this with the Head Code field.

-------------------------------
Paul
EverWeb Developer
User: cecile 12 years ago
Ok, I understood the opposite that you must have a robot txt file to allow the indexation...
Cécile
User: Paul-RAGESW 12 years ago
No, search engines by default will find all websites.

If you think about it, it wouldn't make sense for search engines to require anyone who wants their sites to be indexed to publish a special file.

-------------------------------
Paul
EverWeb Developer


Post Reply
You must login or signup to post.