I had to read this several times last week to get my head around this - now that i do have some idea about the process i understand how awesome the advise is.
I hope you don't mind, but i want to double check if I understand correct.
I already have a huge Identified list would the following be the right steps:
In GSA Options, uncheck all except Identified IN any Project: Uncheck Search engines, and ONLY select Identified from Global Site list
********************
Scenario 2:
I just scraped a huge list of urls do i need to import and identify them first or would the following be the steps:work
IN Options - Temporally changed the Identified path to point to my just scraped url list Unchecked all except identified IN any Project: Uncheck Search engines, and ONLY select Identified from Global Site list
*****************************
IN the project itself, would it be best to create a project using every available platform when i want to run the above import list - or does it not really matter.
If you have an unsorted list, you will need to import it into projects or import/sort in in the global list.
How you break this up depends on whether you want to build links, or build a verified site list as quickly as possible. My choice would be to turn on all of the engines in projects, select all of the projects, and import the list into all of the projects at once. I would use the option to distribute the targets among all projects. I would also break up the list and import it in chunks. SER seems to run better that way. Your LPM will be low, so you will need to be patient and let the list run out. The holy grail is a big, clean verified list that you can use to build real links. You can also increase your threads since the proxies will only be used with Google to check PR.
So unsorted list - need to import \ sort it first. Create project with all platforms selected Use the option to distribute the targets among all projects ( Not needed i guess if u have only the 1 project ) Turn up threads as much as possible
Got it, running it thru the system now and looks like it will have to run overnight and possible for a couple of days, as my list is just over 6.6 m
Just using 1 project Currently running it at 500 threads CPU : 17% Mem: 1.30 GB LPM : 33.28 -- and slowly going up My verified total at start was : 312786 My Verified after running for about 2 hours : 315441 New verified : 2655
I would create a new folder for your list. You are mixing it with your verified list. Create a new folder on your desktop (or elsewhere) and go to global options=>advanced and change the verified folder to your new folder.
I copied my current Identified folder Then i changed the Submitted link to point to the copied verified folder. Unchecked all except Verified.
IN gsa project : Select only global site list : Submitted
So the way i see it, it is using the identify list only and what ever it verifies it adds to my existing verified list -- which is what i want -- trying to grow the verified list. Afterwards i will change all back the way it was, and run a remove duplicate url thing.
Hope i got it right - if not then i will just leave it running because it is building links and verifying them .
Maybe you did this already. i would separate your new verified from existing.1. Point verified (global options) at a new folder. 2. When the new list is finished, point identified (global options) at the new list, point verified at your old list. Point all your projects at identified. The list will fly, and will build the new list to your old list. The only box you ever want checked in global options is verified, and I do mean ever.
Is the whole objective here not to use identified list and to get it verified. So to verify my existing list in Global options leave all as is, the only one that is checked should be Verified
In project select only identified list from Global list
The project will run, and submit the urls in the verified list, when it verifies a link it will add it to my existing verified list, which is the whole objective - is it not ?
At the end of the day, yes, you want a big clean verified list. What you want, first, is rocket fuel for your projects. The rocket fuel is the new list.
SER only builds to files in the folder you have it pointed at. Each engine .txt file has its own unique name. You can't put duplicate engine files in a folder since they would both have the same name. SER only builds to files that it names/creates. You can add files to a folder if you want to, but SER won't write to them.
@Satans_Apprentice After importing our verified site lists into one of the 3 folders (Identified, Submitted or Failed), unchecking all engines and selecting that specific folder in our project to read links from do we have to select all platforms too? Because that verified site lists is holding all platforms links and not only contextual, dofollow links.
When creating dummy project do you build links to one of your actual domain or is random like yahoo.com etc? Because presorted verified list is going to build you huge number and all type of links and that's not want we for our project.
@goonerSo you process raw link files from SB with dummy GSA project on one server and then import that newly created verified site list to your new dedicated server in one of the 3 folders (Identified, Submitted or Failed) and then all of your projects are using just that clean sitelist which in the end builds your old verified list?
I read this a couple of times now but maybe I'm too tired to get it. I don't see the benefit of doing the folder stuff over directly importing targets.
For freshly scraped lists: 1. I import and split a fresh scrape from Scrapebox across 10 projects that post to all platforms 2. My verified list grows due to the verified links from #2 3. I can now use my verified list for other projects
For purchased verified lists: 1. Import the verified list directly into projects, split across 10 spam projects that post to all platforms to clean out the bad links from that verified list 2. as above
EXAMPLE:
1. I import a list of 100k URLs across 10 projects, each project will now have 10k fresh URLs to post to that will all end up on my global verified list.
VS.
2. I import a list of 100k URLs once into the "failed" folder (for example) and then using my 10 projects to post to sites from "failed" - now all those 10 projects would post to the same 100k sites each, making this example slower than the first one...correct? Plus I'd have a bunch of dupes in my global verified list.
The method is about getting the most verified links in total (and the highest LPM). If you split 100k url's across 10 projects each project can only get max 10k verified = 100k verified total If you add 100k url's into a folder, each of the 10 projects can get 100k verified = 1 million verified total
It doesn't matter if projects use the same url's, why should it?
So there are 2 different goals here, get a high LPM (= do the folder stuff) or build up your verified list as quickly as possible (= what I described). Both surely have their purpose.
@johnmiller - Yes you're right. With this method the lists that are being imported are already verified (taken from another server). At least that's how i use the method.
If you are talking about scraping and importing, then 100% it's best to import directly to projects as you said.
Actually, I've changed my stance on the best way to import a sitelist. You get more links faster by importing the .sl file to "identified" in global options, and running from the identified folder.
Comments
I hope you don't mind, but i want to double check if I understand correct.
I already have a huge Identified list
would the following be the right steps:
In GSA Options, uncheck all except Identified
IN any Project: Uncheck Search engines, and ONLY select Identified from Global Site list
********************
Scenario 2:
I just scraped a huge list of urls do i need to import and identify them first
or would the following be the steps:work
IN Options - Temporally changed the Identified path to point to my just scraped url list
Unchecked all except identified
IN any Project: Uncheck Search engines, and ONLY select Identified from Global Site list
*****************************
IN the project itself, would it be best to create a project using every available platform when i want to run the above import list - or does it not really matter.
So unsorted list - need to import \ sort it first.
Create project with all platforms selected
Use the option to distribute the targets among all projects ( Not needed i guess if u have only the 1 project )
Turn up threads as much as possible
I will try this
Thanks again
IDENTIFIED should be unchecked in global options or you will write to that folder and pollute your premium list.
But IDENTIFIED needs to be checked in projects so projects will take URLs from there.
Just using 1 project
Currently running it at 500 threads
CPU : 17%
Mem: 1.30 GB
LPM : 33.28 -- and slowly going up
My verified total at start was : 312786
My Verified after running for about 2 hours : 315441
New verified : 2655
Then i changed the Submitted link to point to the copied verified folder.
Unchecked all except Verified.
IN gsa project :
Select only global site list : Submitted
So the way i see it, it is using the identify list only and what ever it verifies it adds to my existing verified list -- which is what i want -- trying to grow the verified list. Afterwards i will change all back the way it was, and run a remove duplicate url thing.
Hope i got it right - if not then i will just leave it running because it is building links and verifying them .
Is the whole objective here not to use identified list and to get it verified.
So to verify my existing list in Global options leave all as is, the only one that is checked should be Verified
In project select only identified list from Global list
The project will run, and submit the urls in the verified list, when it verifies a link it will add it to my existing verified list, which is the whole objective - is it not ?
I read this a couple of times now but maybe I'm too tired to get it. I don't see the benefit of doing the folder stuff over directly importing targets.
For freshly scraped lists:
1. I import and split a fresh scrape from Scrapebox across 10 projects that post to all platforms
2. My verified list grows due to the verified links from #2
3. I can now use my verified list for other projects
For purchased verified lists:
1. Import the verified list directly into projects, split across 10 spam projects that post to all platforms to clean out the bad links from that verified list
2. as above
EXAMPLE:
1. I import a list of 100k URLs across 10 projects, each project will now have 10k fresh URLs to post to that will all end up on my global verified list.
VS.
2. I import a list of 100k URLs once into the "failed" folder (for example) and then using my 10 projects to post to sites from "failed" - now all those 10 projects would post to the same 100k sites each, making this example slower than the first one...correct? Plus I'd have a bunch of dupes in my global verified list.
What am I missing?
If you split 100k url's across 10 projects each project can only get max 10k verified = 100k verified total
If you add 100k url's into a folder, each of the 10 projects can get 100k verified = 1 million verified total
It doesn't matter if projects use the same url's, why should it?
So there are 2 different goals here, get a high LPM (= do the folder stuff) or build up your verified list as quickly as possible (= what I described). Both surely have their purpose.
If you are talking about scraping and importing, then 100% it's best to import directly to projects as you said.
In the project all engines unchecked, Use url from global list checked and submitted folder is the only one checked. Where to submit: all checked.
I did delete url cache and url history.
Now when I start it says "Loaded 0/0 Urls from sitelist".
What I am doing wrong?