The Only World-Standard SEO Software

SEO and Internet Marketing Forum
Welcome, Guest ( Login | Register )

Website Auditor crawler/bot name
New Topic Add Reply
Page 1 of 1 [ 4 posts ]
Author Message
fulafaq@astd.org

Newborn

Newborn
Posts: 1
Posted: Thu Jun 12, 2014 9:05 pm

Hi,

Can some one tell the BOT name of WA crawler.

I am using WA to find broken links. But the crawler is scanning some links from a script which result in 3-5 broken links in each page.

We'll have this scripts hidden from the crawler if we know the name.

Thanks,
Faheem

LinkAssistant

User avatar
Site Admin
Posts: 5588
Posted: Fri Jun 13, 2014 2:24 pm

Faheem.

Website Auditor uses random user-agents, however you can specify your own under Preferences - Search Safety settings - User Agents, this way you will know which queries have been sent by the software.

robbie3

Senior

Senior
Posts: 26
Posted: Sun Mar 05, 2017 11:43 pm

Which is the name of spyglass bot

I want to block it from my site.

This information should be public like moz or ahref and I cant find it :@

LinkAssistant

User avatar
Site Admin
Posts: 5588
Posted: Mon Mar 06, 2017 7:04 am

SEO SpyGlass is using WebMeUp database, you may find the information here:

http://webmeup-crawler.com/

Display topics from previous:    Sort by    
New Topic Add Reply
Page 1 of 1 [ 4 posts ]
All times are UTC

Jump to:  


cron