I can't figure out why my PI is CONSTANTLY crashing now. It ran a 3.5M Scrape just fine the other day. Now i'm trying to run one that I split to only 1M and any random amount of time it just completely hangs and can only be closed by CTRL+ALT+DELETE which totally kills your current status. I've tried randomizing the urls before loading to PI. I've tried trimming URLS longer than 150 characters. I've tried all kinds of shit.
This app very very badly needs an auto-save function like SER and the ability to use proxies. When it keeps crashing I don't wanna have to keep hammering the same damn domains over and over and get a abuse report or 403 etc.
Currently my only way of restarting where my lists left off is doing only 1 at a time. When it crashes import all the platforms it recognized and the unrecognized to ScrapeBox. Remove Duplicate Domains. Select the original Scrape file and use "select url list to compare" function in ScrapeBox against all my platforms recognized and unrecognized, which is slow as shit with huge files. ScrapeBox 2 ain't any better at it either with it being 64bit.
EDIT: I've also tried running compatibility mode. Don't matter. I've tried reducing threads drastically. Tried ALOT. Thinking it's a problematic url or urls but no way to identify it that I can think of.
EDIT2: I'm running this only PI on a separate DEDI where SER isn't running. Ideas?