I would never share/swap or even sell my site list. Do you have any idea how much money and time I spend every month to build my site list? Then you expect it to be shared for free? Do you know how expensive proxies are for scraping google each month?
I would never share/swap or even sell my site list. Do you have any idea how much money and time I spend every month to build my site list? Then you expect it to be shared for free? Do you know how expensive proxies are for scraping google each month?
You must be nuts! lol
This is off-topic to the thread, wondering if you built your own script for scraping or do you use Scrapebox?
hello guys i see that the main issue is to find a list. is that means that their is no reliable list provider in the market ? becuase if thier is not then why using GSA
The ability to scrape and add these sites to the software is a unique feature of this software. This means there is no limit to how big your site list can get.
I've over 10,000 blog comment sites, over 30k redirects, over 100k indexers and over 10,000 contextual/profiles lol That's what you can get if you scrape your own targets. I've still not finished sraping all the different variations yet. It would be nice if these list sellers had these sorts of numbers in their lists, but they simply don't.
Many of these sites are still unspammed as it's only me posting on them. They will have a much longer shelf life than paid lists that's shared amongst hundreds of users.
The ability to scrape and add these sites to the software is a unique feature of this software. This means there is no limit to how big your site list can get.
I've over 10,000 blog comment sites, over 30k redirects, over 100k indexers and over 10,000 contextual/profiles lol That's what you can get if you scrape your own targets. I've still not finished sraping all the different variations yet. It would be nice if these list sellers had these sorts of numbers in their lists, but they simply don't.
Many of these sites are still unspammed as it's only me posting on them. They will have a much longer shelf life than paid lists that's shared amongst hundreds of users.
Very impressive. Did you have to add non-default engines to SER or did you manage to collect this all these sites using the already existing engines?
The contextual/profiles are mostly due to @cherub and his new engines: gnu, dwqa, wowonder and git alikes engines mixed in there too.
But aside from these, it's using the default engines. There are over 400 different cms supported. I just pick one and focus on scraping with different footprints and keywords, then move onto the next one. It's a never ending exercise.
I could do it a lot quicker if I had some affordable proxies. That's my only hold up right now. Have to pace myself due to cost of proxies and data.
I see, yes that is why I set up a custom 4G mobile proxies farm, it's small at the moment, but I can rotate these proxies (change IP) as much as I want (although it takes roughly 12 seconds to switch ips).
For contextuals, I finished building custom software that communicates with SER, and what it basically does is generate entire static websites (like a blog). My software can then publish these sites to static hosting sites (for example S3).
Its really nice in the sense, that SER will then verify the links, and my Tier 2 project can take over and send links to these sites. This is what I love about GSA SER. It is extensible and allows to maintain almost full automation.
I'm running a test at the moment, but 28 sites out of 60 are indexed after a week (not really sure if that is good, but it is something at least).
Will look into cherubs engines, could save some time for me.
Will start to focus on building custom lists as well, butthink I need to start coding... think it would be nice to automate as much as possible for scraping as well.
I've tried mobile proxies with rotation. They worked pretty good but have to keep threads low to avoid ip bans. I've had scrapebox at 500 threads and it was scraping over 1000 urls/sec, but I was being charged $0.50/GB with that proxy provider, so not something I could sustain for long.
Sounds like a nice solution you have there. 28 out of 60 is pretty goood for 1 week. Google can take several weeks to index sites. They will eventually be 100% indexed once google find all your links.
The custom engines from @Cherub are really good. Will expand your site list very quickly. I'm waiting for him to make some more new engines.
I have set fixed plans at 20GB at the moment, but could up the sim card plans to "unlimited", lets see if that will be needed soon Yeah its not bad, there are around 30 themes in the software, so there is good variety, but there are still things I need to fix. When the sites are created they are fresh, so it will probably take some time for them to power up the end site as they need link juice from lower tiers.
I've tried mobile proxies with rotation. They worked pretty good but have to keep threads low to avoid ip bans. I've had scrapebox at 500 threads and it was scraping over 1000 urls/sec, but I was being charged $0.50/GB with that proxy provider, so not something I could sustain for long.
@sickseo I'm scraping similar figures using IPv6 proxies with Scrapebox on Google only. Costs $50/m, based on # threads only. Check the botmaster forum for vendors. You can even change the country US, UK, Germany, Switzerland, ... whenever you want. GSA, BHW forum offers are just resellers.
Same applies for 3rd party captchas, etc.. There are many resellers taking huge benefit of people's political views and laziness to translate.
calix I would say that's pretty great indexing rate so far.
I agree, I love how SER allows for full customization of many things. It's really on user to tweak it to their needs. The tools are all included. I love to tinker and make custom solutions myself. Yours sounds like a really interesting project.
Thank you. Lets see, hopefully with enough patience and work it will add value to the overall strategy.
I've tried mobile proxies with rotation. They worked pretty good but have to keep threads low to avoid ip bans. I've had scrapebox at 500 threads and it was scraping over 1000 urls/sec, but I was being charged $0.50/GB with that proxy provider, so not something I could sustain for long.
@sickseo I'm scraping similar figures using IPv6 proxies with Scrapebox on Google only. Costs $50/m, based on # threads only. Check the botmaster forum for vendors. You can even change the country US, UK, Germany, Switzerland, ... whenever you want. GSA, BHW forum offers are just resellers.
Same applies for 3rd party captchas, etc.. There are many resellers taking huge benefit of people's political views and laziness to translate.
Sorry, stupid question, do you mean botmastersupport.com (just googled it)? Where can I find "Botmaster forum"?
Hi, I hope we can exchange lists, I sent you a message.
I haven't received your TG information. Currently, I have collected information from several mainstream suppliers on the list. Which one are you exchanging?
Comments
You must be nuts! lol
I've over 10,000 blog comment sites, over 30k redirects, over 100k indexers and over 10,000 contextual/profiles lol That's what you can get if you scrape your own targets. I've still not finished sraping all the different variations yet. It would be nice if these list sellers had these sorts of numbers in their lists, but they simply don't.
Many of these sites are still unspammed as it's only me posting on them. They will have a much longer shelf life than paid lists that's shared amongst hundreds of users.
But aside from these, it's using the default engines. There are over 400 different cms supported. I just pick one and focus on scraping with different footprints and keywords, then move onto the next one. It's a never ending exercise.
I could do it a lot quicker if I had some affordable proxies. That's my only hold up right now. Have to pace myself due to cost of proxies and data.
For contextuals, I finished building custom software that communicates with SER, and what it basically does is generate entire static websites (like a blog). My software can then publish these sites to static hosting sites (for example S3).
Its really nice in the sense, that SER will then verify the links, and my Tier 2 project can take over and send links to these sites. This is what I love about GSA SER. It is extensible and allows to maintain almost full automation.
I'm running a test at the moment, but 28 sites out of 60 are indexed after a week (not really sure if that is good, but it is something at least).
Will look into cherubs engines, could save some time for me.
Will start to focus on building custom lists as well, but think I need to start coding... think it would be nice to automate as much as possible for scraping as well.
Sounds like a nice solution you have there. 28 out of 60 is pretty goood for 1 week. Google can take several weeks to index sites. They will eventually be 100% indexed once google find all your links.
The custom engines from @Cherub are really good. Will expand your site list very quickly. I'm waiting for him to make some more new engines.
Yeah its not bad, there are around 30 themes in the software, so there is good variety, but there are still things I need to fix. When the sites are created they are fresh, so it will probably take some time for them to power up the end site as they need link juice from lower tiers.
Same applies for 3rd party captchas, etc.. There are many resellers taking huge benefit of people's political views and laziness to translate.