Skip to content

ProxyScraper is now a Mozscape / Majestic Scanner as well

SvenSven www.GSA-Online.de

With the latest Update of this tool you also have a tool which let's you resolve all data from Mozscape and Majestics.

image

More about it here: http://docu.gsa-online.de/proxy_scraper/url_metrics

Comments

  • NetMajomNetMajom https://buywebgraphic.com | skype: nszmmp
    Its amazing, but still getting data message.... It working very slow....
  • SvenSven www.GSA-Online.de
    All depends on your proxies and of course at startup it is a bit slow as it has to find matching proxies that will work. Once found it caches them and speed should increase over time.
  • vignesh676vignesh676 Mumbai,India
    Great Feature! Thanks.
  • Do we need our API? for Majestic and Moz?
  • s4nt0ss4nt0s Houston, Texas
    edited September 2015
    @Anonymous - Nope, just load URLS. 
  • SvenSven www.GSA-Online.de
    I don't know what you get with api keys from majestic. But you get the major values witht he tool and all from Moz.
  • Nothing loads. Tried from Clipboard and file. Nothing shows up on the software.
  • SvenSven www.GSA-Online.de
    I guess the file you import is not in ansi format and so it will not show.
  • How about loading from clipboard?
  • SvenSven www.GSA-Online.de
    that is the same story if the clipboard data is not ansi...however try latest version please.
  • wao . its amazing ... now my work will be more easy..
  • can you pull referring domains count as well from majestic?
  • SvenSven www.GSA-Online.de

    the following can be extracted from Majestics without any API key:

    TrustFlow URL
    TrustFlow SubDomain
    TrustFlow RootDomain
    CitationFlow URL
    CitationFlow SubDomain
    CitationFlow RootDomain
    ExtBackLinks URL
    ExtBackLinks SubDomain
    ExtBackLinks RootDomain
    RefDomains URL
    RefDomains SubDomain
    RefDomains RootDomain
    
  • nice, is there any possibility to do the same for ahrefs?
  • SvenSven www.GSA-Online.de
    do they offer some kind of free access?
  • no. Ahrefs wont.
  • KaineKaine thebestindexer.com
    This is possible use this new option with Ser ? For replace PR for exemple ?
  • SvenSven www.GSA-Online.de
    a bit difficult as you need to have a lot proxies to have this working fluently.
  • KaineKaine thebestindexer.com
    Yes think of that after and is not easy. Just see many com go in this direction before and now you have solution :)

    But surely anonymous or élite proxy is ok for that.

  • Note: Import to URL Metrics Scanner via File or Clipboard without "www." at the beginning not work.
    "site.com = not work, "www.site.com" = work.
    And yes, file must be in ANSI format.

    P.S. Please place note in docs for import or just fix this please.
  • SvenSven www.GSA-Online.de
    well nothing to fix as site.com is no URL but a domain ;)
  • edited November 2015
    Hey Sven,

    When i load the URL as http://www.domain.com , it never checks URL Trust Flow - http://prntscr.com/93gldo
    If i load it as http://www. does it check http:// (No WWW version) ?

    Thanks
  • Same thing, I never get results at all for URL Trust Flow and URL PR. EVER
  • SvenSven www.GSA-Online.de
    @nemanja87 no, it checks with full URL.
  • Hey Sven,

    Thanks for a fast reply.

    It's working good for me as is, root and subdomain metrics are good start, but sometimes URL has much higher value (no www version is higher in about 5-10% of cases, and www version returns 0 in that case, so you would think its a dud, even if its quite good domain)

  • @nenmaja87

    That happens if you directly check with Majestic as well... it comes down the the linking i think. Not sure. 


    ive had different metrics reported with posts from www.example.com http://example.com and example.com
  • Sven, problem with www.site.com vs site.com was in interface design. When you don't know how import work you try import file without www. and get nothing in return... no error, no warning, just nothing. No info in docs. Just add notice or window popup with warning that file format incorrect.
  • Here is some suggest to improve URL Metrics.
    1) add in right mouse click "Reset checked status" (or Reset Status). Sometimes only checked Majestic data but Moz skipped or some problem with proxy. But status of domain moved to "Checked" (green icon). But need all data from Majestic and Moz. If have big list of URL/Domains is quite pain to delete and run checking again and again.
    2) add schedule for checking data. When big list URLs complete you have OK (green icon) and non-checked domains (red icon). So you click Start button again and again until get all data. But idea to leave URL metric to do it automatically (on night). So fresh proxy schedule + URL Metric schedule = win
    3) combine 1) and 2) to get full Moz+Majestic data automatically. It mean import list of URLs/Domains, press start and URL Metric do job UNTIL all Moz and Majestic data for ALL domains is complete. Not partial only Moz or only Majestic but all for all columns.
    4) add Majestic note for columns like Moz. If it too long just add to mouseover note for column.

    Thank you
  • SvenSven www.GSA-Online.de
    ok, next version will give you a warning/message about imported urls.
  • SvenSven www.GSA-Online.de

    1) added for next update

    2) it's repeated at the end unless you stop it in next version

    3) I try my best

    4) I thought thats already there? Please explain whats missing here.

  • Sven,
    0) import info/warning popup works great! thank you
    1) Retry at end works great. thank you
    2) same as Retry
    3) same as Retry. Thank you
    4) just ignore or simply add when mouseover column same as "MozRank:" - "Majestic:". It good futureproof if you add ahrefs or other SEO checking API.

    Interesting note: played with URL metrics for day and notice that if you add sites for checking when Proxy Scraper worked for several hours usually you stuck with URL metric checker: no data, tons of "proxy seems down", "no valid init reply (500 internal server error).
    But if you restart proxy scraper and try again URL Metrics then at start you'll get full data from majestic and moz in several minutes (for not large list 20-60 sites) but then again stuck. With this trick i did 300 sites (divided to 6 x 50 sites list) in less then hour. But if i leave 300 sites in URL metric for hours in most cases it will not finish.

    Anyway thank you for great update Sven.
  • Hello,
    I'm trying to use the Metric Scanner. Lots of problems with it.The scraper in the beginning worked fine for about 2 hrs. and then hasn't worked right since.It stops working,won't scrape proxies and when it test proxies that's all it does is test and when it finishes there's no proxies out of 30,000. Still stopped. The automatic proxy search is enabled. I've loaded it with proxies from SB and the Metric scanner starts working again but the error box pops up and must close application. I need help!!! I just read this last comment. I have a million urls to clean.
  • SvenSven www.GSA-Online.de
    A restart doesn'T help?
  • I'm currently trying the trial version of the Proxy Scraper to decide wether I should go for a Majestic Platinum subscription or wether Proxy Scraper can handle bulk checking by itself. Everything seems to be going ok, except the Trustflow URL data doesn't get picked up (which is most important for me).

    Is this a limitation in the trial version or is there some kind of bug ATM?
  • SvenSven www.GSA-Online.de
    Yes it's a 5 day trial. However TF is something which is delivered together with the other data and should be available all the time once MOZ replied.
  • Hmm, it's definitely not picking it up. Every other metric is working fine, but the URL TF column remains empty. Log doesn't show anything either and neither does the exported .txt file.  :(


  • SvenSven www.GSA-Online.de
    Oh wait, are you sure that this URL has been crawled by MOZ at all? Usually there is just data for domain/subdomain.
  • edited March 2016
    TF is a Majestic metric, but yes, 100% sure. It's happening with thousands of URLs and both the Majestic Chrome extension and their direct database show the stats.  

    I think something might have changed in how Majestic handles TF URL retreivals.

    Edit: Besides, the TF URL column should say ''0" like it does with the root/subdomain TF/CF stats when not crawled/no equity. 
  • SvenSven www.GSA-Online.de
    ok found the issue now and will fix it on next update
  • edited March 2016
    Thanks, I'll make sure to grab a full copy then. :)

    Seems to be a way better alternative than PRJacker (which seems buggy as hell lately). Sure, it's double the price, but with the reputation of Sven you're guaranteed lifetime updates / quick bug fixes and have the added proxy scraping (even with PRJacker you'll need proxies anyway) and / SE parser benefits. 

    I know you don't like it, but you should promote this future a bit more, Sven. I was pleasently suprised about the speed and number of metrics that can be checked.
  • SvenSven www.GSA-Online.de
    It's as always a marketing issue, we have very little time for this.
  • edited March 2016
    I have tested it and it doesn't show the majestic TF for the url - it shows the CF but not the TF
  • SvenSven www.GSA-Online.de
    again, I will fix it in next update. I haven't released that yet.
  • @Sven not sure if others are seeing this, but when I run 1.61 it doesn't return any Majestic metrics, but when I revert to 1.60 it does...
  • SvenSven www.GSA-Online.de
    hmm working for me. Did you choose any Majestic data in settings?
  • It's set to pull back domain and page TF & CF
  • spunko2010spunko2010 Isle of Man
    @Sven open it up to affiliates to promote. Yes, you need to hand away more of the money but more will come in and there's no work needed on your part.
  • SvenSven www.GSA-Online.de
    @spunko2010 we have an affiliate system ;) let me know if you want to sign up and I send the link in pm.
  • spunko2010spunko2010 Isle of Man
    @Sven I don't do this type of affiliate promotion any more but lots of people do, is it a private scheme? Maybe make it publicly available.
  • @spunko2010

    I don't think he wants it public--most people already know about SER.
  • SvenSven www.GSA-Online.de
    the problem with public affiliates is that they don't really care about it. If I give the link to sign up to anyone then I hope he really puts some effort into it.
  • spunko2010spunko2010 Isle of Man
    Yeah, I understand that, but I still think maybe something like a link in the footer to a page saying something like "We have a private affiliate scheme, tell me about your ideas of how to promote the software" or something. As it stands now, it looks like there isn't one at all. Just my 2 cents anyway :)
  • I never managed to get stats for majestic URL as well, I purchased this tool September 2015, and till now I never got any data.
  • SvenSven www.GSA-Online.de
    Anonymous how many proxies do you usually have here?
  • 2000 proxies, we discussed that via PM in past, it's just not working that's all and as you see am not the only one with this problem.
  • url metrics scanner in trial version not work ?
  • SvenSven www.GSA-Online.de
    it should work but recently it seems a lot proxies are blocked on the providers.
  • hi Sven, v1.65 wont work for me now. I have add some domain but it only scrape PR data only.
    should I reinstall it?

    Besides, could you add the function to check the domain availability?  and some web 2.0 availability? such as tumblr
  • SvenSven www.GSA-Online.de
    well it's a proxy scraper  in first place...I don'T know what you mean with web 2.0 availability
  • I think he means he wants to scrape for available web 2.0 properties that have expired like the scrapebox addons.
Sign In or Register to comment.