The Only World-Standard SEO Software

SEO and Internet Marketing Forum

SEO PowerSuite Christmas Sale 2018

Welcome, Guest ( Login | Register )

Robots Coding format important in our website coding side?
New Topic Add Reply
Page 1 of 1 [ 3 posts ]
Author Message


Posts: 2
Posted: Thu Jul 23, 2015 11:37 am

meta name= "Robots" Content ="Index", follow> Why use this coding in website its important?



Posts: 10
Posted: Mon Aug 10, 2015 9:01 am

The robots meta tag is not the same as the file called robots.txt You should use these two together. Both are used by the seach engines like Yahoo and Google. If your tag indicates index, follow then it shows search engines that he should index current & remaining pages of the website.

Cityweb - SEO Services Pune | Online Marketing

priyanka prasad


Posts: 108
Posted: Thu Jan 11, 2018 2:06 pm

Website owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
It is important for our website because before a search engine crawls your site, it will look at your robots.txt file as instructions on where they are allowed to crawl (visit) and index (save) on the search engine results. Robots.txt files are useful If you want search engines to ignore any duplicate pages on your website.

Display topics from previous:    Sort by    
New Topic Add Reply
Page 1 of 1 [ 3 posts ]
All times are UTC

Jump to: