Import Lists
@Sven or anyone else who me be able to help - Is there a way to set up SER so that when it has finished processing one list of potential sites, it automatically loads the next one?
The reason I ask is that I've been v busy scraping lately and I've got something like 40million *potential* pages I can drop a link with SER. I've split them into lists of 500k, but even so that's 80 files I've got to import and when I've been importing these into a couple of campaigns I've created purely to process these links, it seems to slow everything down.
It might not make a difference to the speed, but it would be good if from within a project you could set a directory where it imported link lists from and then I could spilt them into smaller files of about 10k sites.
Thanks
The reason I ask is that I've been v busy scraping lately and I've got something like 40million *potential* pages I can drop a link with SER. I've split them into lists of 500k, but even so that's 80 files I've got to import and when I've been importing these into a couple of campaigns I've created purely to process these links, it seems to slow everything down.
It might not make a difference to the speed, but it would be good if from within a project you could set a directory where it imported link lists from and then I could spilt them into smaller files of about 10k sites.
Thanks
Comments