Which search engines to use?

pietpatat

I created this thread so we can brainstorm and discuss about what search engines to use when scraping for new results. Because a lot of search engines use the same results and selecting these se's will result in a lot of duplicate results and this will make GSA not run as effecticly as it could be running.

@ron already stated:
 "Google results are identical for a number of America/English
properies like the Cook Islands, Bermuda, etc. So you are duplicating
efforts when you choose those.

Startpage pulls Google data, so that is a dup. Of course, Bing = Yahoo, but a question might be which is better, Bing or Yahoo."

So we already have a couple of se's not to use.

Besides that @GlobalGoogler is also doing an experiment on this here: https://forum.gsa-online.de/discussion/1225/drastic-efficiency-improvements-part-2-se-logic-module/p1, which will probably also result in some pretty interesting results.

I'm also wondering what search engines other users have selected? How many? All from different countires or all international?

I also have a question for @sven about selecting Google from different countries in GSA: If you are from the netherlands you will automatically be redirected to google.nl if you go to google.com. This is probably the same with GSA, I have my VPS in germany with german proxies, so if I have google.com selected in GSA it will probably redirect me to google.de right? If so, this can be fixed by adding /ncr ( no country redirect ), so going to google.com/ncr will show the google.com results and not redirect GSA to google.de. So my question is does GSA already add /ncr and if not can this be added?

Brandon

I select by Country > United States.  That gives 14 search engines.  I don't know if this is best, but it's how I set up campaigns.

sonic81

Hi Brandon.  How many keywords are u using?  I find if I don't select a lot of search engines the campaign links submitted drops drastically after a few days (I suspect due to not finding new sites to post)

Trevor_Bandura

I noticed that when I click on a log entry for search results, lets say it's for Google MX, when I go to that search result it's showing pages from Mexico and this in the URL (cr=countryMX). Now if I remove that cr= from the URL, it seems to give more results at times.

How can I change it so when ser makes a search, the cr=?? is not added to the search query?

seagul

6 random Googles so that you have different SEs than other projects. This helps prevent IP bans. Private proxies are a must

Ozz

i recommend to use "Google RND" as well as it will query all the google hosts directly. if you are banned on one host you have a chance that you aren't banned on some of the other hosts.

<How can I change it so when ser makes a search, the cr=?? is not added to the search query?

you are able to create custom SEs. just open "se.dat" in your installation folder with an editor, create a file called "user_se.dat" and put that into your appdata folder of SER. now copy a google SE from se.dat to user_se.dat, rename it to [Google Custom] and modify the "cr=" operator to the "url=" line.

save user_se.dat, restart SER and you'll find your custom SE in the selection window.

Trevor_Bandura

I did what you said but I can't see the new SEs from the new file I created.

The DAT file I created is in the same folder as the original se.dat file, is this correct?

Ozz

no, it needs to be in your %appdata% folder. something like C:\Users\Admin\AppData\Roaming\GSA Search Engine Ranker

Trevor_Bandura

ahhh okay.

Got it, that worked. Thanks.