Anyone have a way to clean site lists?

jman75

Is there a way to clean site lists I'm trying to get my lpm up I'm running 860 threads not scraping proxies are still good , My lpm is 20 using my site list of 44k urls. How do I take the successful sites and put them into a new list and jus run those sites. I get alot of messages in gsa tab saying no engine matches which in my theory is just wasting time on trying to build a link there I would like to take my good sites make one good list can anyone help willing to pay a little pm me ?

Sven

what you can do is

a) use options->advanced->tools->clean up

b) import all site lists into a project (turn all engines on you are interested in), delete all site lists and start it. All site lists will be rebuild.

matt_rankman

If you use a), don't abort it or it'll remove everything from site list except the ones that were checked by this tool. (Correct me if I'm wrong). Just happened to me, thankfully I had backup.

jman75

@matt_rankman @sven how do I export sites that work causr I would like to get rid of sites that dnt work make a new list and jus add that list to my projects

jman75

Im a little confused what do you mean by turn on engines ? Im not scraping from search engines jus using sites

gooner

@jman75 - Sven meant ensure you check the correct engines in the project you import the list to.
Best to check all engines if you are importing an entire sitelist.

Justin

Here's what I would do. 1) I would go to your global options, click tools, export your verified sitelist as a .sl. 2) Create a new project not using search engines or posting from global site lists. 3) Use very generous settings in setting up this project. In global filter option, increase the max website to download size to something like 10mb, increase your HTML timeout to 180 seconds, and decrease your threads down to like 25% of what you would normally run under max efficiency. 4) Open the folder containing your verified sitelist and delete it completely. This folder will now be empty. 5) Go to the project that you just created right click>Import Target URLs>From File and choose the exported verified site list that you created in the first step. Alternatively to this, you can create a new folder for SER to read from in the global identified folder, import your exported verified sitelist to your newly created identified folder, and then when telling where your newly created project to get URLs from, choose to Import Target URLs From Site Lists and choose your identified site list. This will sequentially go through each and every target in that list rather than randomly choosing targets like it would do if you told SER to post from global site lists within the project options, thus missing a lot of potential targets.

Make sense?

jman75

I see where do I check site engines I been using gsa for a month

jman75

@gooner

gooner

In the project options, you select (check) the engines you want to use.

jman75

Wiki, blog comment , social bookmatk are thosr engines you mean @gooner

Justin

These are what people mean when they refer to engines.

gooner

@jman75 yes, just as @justin posted above ^^

jman75

Ok got you @justin thanks going to try it latet I get alot of no engine matches I have everything checked except video and document sharing also have facebook and youtube sites in my list like to clean it up ill let you know how I make out later. My global options is that same place I find proxies, captcha etc in that tab ?

jman75

@gooner @ justin do yous have a good site list I can buy my lpm is 20 right now tryinh increasr that

gooner

@jman75 - We do sell lists, although we are sold out at the moment. A new one will be available in a week or so.
For more info see: https://forum.gsa-online.de/discussion/9464/new-two-massive-verified-sitelists-50-000-and-75-000-from-serlists-com#latest

jman75

@gooner im signing up for it now