Skip to content

How can i better my stats and performance? Settings included

reupreup
in Need Help
I've done some research and adjusted my settings but i would like some advice on how to better my performance, rght now my LPM is at 2.75. I would like that to increase if anyone has some minor or major tweaks or can point out what i am doing wrong. or is this a good LPM to have if i'm submitting to PR3+ on first tier and PR2+ on second tier, etc...?

Would really appreciate some pointers, thanks.

I have 200 threads with a HTML time out of 180 and 60 seconds between each search query.

Proxies
I have 19 Dedicated proxies - 10 Semi dedicated

Projects
Running 3 projects 
1 has 5 Tiers 
1 has 2 tiers
1 has 3 tiers

My Hardware
- Windows 7
- 10 GB Memory
- 920GB HD

Submission
Here are my Submission settings
image

Captcha 
Using GSA CB as primary DBC as secondary
image

Indexing
I have GSA indexer off for now. Sending to LinkProcessor with only PR2 or higher. Should i adjust this?
image

Filter Settings
image

Advanced Settings
I have only ' Verified ' selected
image

Options for my first tier
Originally i was targeting specific countries, should i switch that to 'language'?


image

Comments

  • fakenickahlfakenickahl
    edited May 2014
    Are your proxies getting burnt for pr checking? if they are then they're going to slow you down a LOT.

    Don't use proxies for verification and check "skip for identification"

    Why not have CB try more than once to see if it can solve a captcha for free before you spend money on it?

    Why would you only send verified links with PR 2+ to your indexer? Makes no sense to me.

    You can increase the "maximum size of a website to download" value.

    Personally, I'd check "try to always place an URL with an anchor text in descriptions/comments" and "continuously try to post to a site even if failed before".

    You are using a LOT of search engines that are never returning any results for your search queries. You need to look at your log and figure out which search engines are never giving you any results while scraping.
  • reupreup
    Thanks for kindly answering me! @fakenickahl

    Yes i noticed they aren't effective as they should. I adjusted my settings, to your suggestions. 
    - I de-selected 'verification' and selected skip for identification.
    - adjusted the Captcha settings, I took into account what you said about why not let CB retry rather than pay DBC
    - disabled the submission for PR2+ index 
    - adjusted the 'try to place url with anchor' and 'continously try to post to a site even if failed'
    - for the "maxium size of website to download" ? Is 5mb good enough?
    - In the log file what specific message am i looking for? "download failed' ?

    I appreciate your time and knowledge on this, i'm new to GSA and would love to use its power to the full capacity. 

    Again, your help is greatly appreciated. 
  • fakenickahlfakenickahl
    edited May 2014
    You're welcome mate.

    - for the "maxium size of website to download" ? Is 5mb good enough?
    I believe I read a few weeks ago that the average web page is about 1.5mb in size. Some people are even using 50 mb, but you should be aware that this can cause a lot of RAM usage and thereby slowing you down. You should be good until you start getting those damned "out of memory" errors.

    - In the log file what specific message am i looking for? "download failed' ?
    You need to look at your log when SER is scraping and taking note of those search engines that are never returning any results for your search queries while not being burnt. You can for example make a bunch of the same search queries yourself in your browser and see what happens. Personally I would never bother with all those tiny search engines and instead focus on the larger search engines while using multiple versions of them. For example for Google choosing the US, Great Britain, Australia, etc. When your proxy gets burnt on google.com, it doesn't get burnt on google.co.uk. You could do the same for all the other major search engines that are not returning results for your search queries.
  • reupreup
    Sweet! I just ran my projects with the adjusted settings and my LPM skyrocketed from 2.75 to 6.05 that's a good improvement so far

    - I left the size of the website at 2mb which is more than average, PC seems to run at normal speed not sluggish.

    - When your proxy gets burnt on google.com, it doesn't get burnt on google.co.uk. i did not know this, it's good to know. I have gone back and adjusted the settings and will see what improvements come from this. 

    Thanks i appreciate your time and knowledge  

    let me buy you a beer? :) ha
  • fakenickahlfakenickahl
    I'm glad you're seeing some improvement, but have you made sure your proxies aren't burnt for PR checking yet? You can test all your proxies against google PR checking in SER to figure out if they are. If all your proxies are burnt for PR checking then you will not be able to use your scraped urls for any project with pr filters.

    To make SER run faster and faster you have to just babysit it a little to watch what it's doing and play around with the settings to see what happens.

    It's cool mate, I enjoy helping when I have the time :)
  • reupreup
    Thanks. i just ran a quick test and they all passed successfully. I will try and fiddle around and see what minor tweaks i can make to my options to improve it more. 

    This was a great stepping stone to getting my settings to where they should. i appreciate it your time and help. thanks @fakenickahl
Sign In or Register to comment.