The right way to import verified site lists...
![Satans_Apprentice](https://forum.gsa-online.de/uploads/userpics/143/nUJTLNJQ8PS7L.png)
in Need Help
Here is my opinion, based on experience - do not use the import site list function. It works just fine, but it could cause issues later. Over time many links go bad, and they sit in your verified list until you clean them through one of several means in the SER software (Reverify, Cleanup, etc).
Here is the problem with using the import site list function - if some of the links in the verified site list are bad (and they will be), you can't build a link to them. Furthermore, you can't purge them from your verified list since you never built a link with it, a requirement for any cleanup. Over time, your verified list will accumulate more bad links. If you delete projects, and those links go bad, they will pollute your verified list as well. If you are running off of your Verified site list, bad links will kill your LPM. I get over 100 LPM with a clean list, and under 20 with a dirty list.
Here is my recommended way to import a sitelist:
1. Save the new sitelist to its own folder (duh)
2. In Global Options=>Advanced, point your "Submitted" folder via the dropdown menu at your new sitelist. Be sure the "submitted" box remains unchecked when you are done. Otherwise, you will pollute your nice new list.
3. Make sure the "verified" box is checked.
4. In your projects, click the checkbox "On" for options=>submitted (under the search engine selections). Identified, Verified and Failed should be unchecked.
5. Uncheck all of your search engines. You only want to run off of your list.
Start SER and run it until your LPM collapses. You will be amazed at your LPM. When your LPM finally dies, go back and undo the steps above.
SER will read and build links from your submitted list (your new list), and save them to your standard verified list.
Comments
Nice one @satans_apprentice - Good method.
Why dont you just import>sort in and identify then it removes bad links?
Today is my first day using this technique and i'll end up with 140k verified maybe, that would never be possible if i spent hours identifying and sorting the list first - That function is a waste of time IMHO.
:-?
if you identify and sort in, if the link is bad and it cant build a link there, then it will be disregarded, so the same result occurs, just slower. ( i can see that now)
And because your projects are not holding thousands of targets you get decreased memory usage, which means you can increase threads and get even higher LPM.
So from my understanding,
1) I move my existing "verified" list out of GSA and saved in under other name like "to be verified again"
2) The "verified" list in GSA is empty now. Redirect the "submitted" port to my "to be verified again" list
3) "Submitted" is unchecked so that the list won't polluted when GSA running, and "Verified" is checked so that GSA can write data into it.
4) Create a project and set it to pull url from "Submitted" only.
5) Wait for it to finish and now I have a filtered verified list.
Correct me if I'm wrong. Thanks!
FengLi
This system really works best if you are getting fresh verified lists from somewhere, either buying them or producing them yourself using a second installation.
How you do that is up to you but the morale to the story is old verified lists = low LPM.
I had a 500k unique domains verified lists, cleaned as far as SER could clean it and my LPM was around 70. I switched this for a new verified list of 30k and my LPM shot up to 280.
Do I uncheck them all or leave identified check? You say both.
Under Global Options...only thing checked is VERIFIED. The folder with my premium lists is in the IDENTIFIED section but that section is NOT checked.
Then under my project, i deselect all search engines but check IDENTIFIED so the project will check whatever is in that location.
Correct?