BuddyPress - Contextual Article link creation
I've been trying to build up contextual links, and have been scraping per platform. I've had variable success using BuddyPress, so did some troubleshooting to see if there was anything I could do to improve the process. Listed below is an overview of the process as well as some interesting findings:
SETTINGS
- Used Gscraper and custom footprints for BuddyPress - got around 6,000 unique domains to test with SER.
- ran the list on 3 different dedicated servers (stopped other projects while doing troubleshooting)
- 50 semi-dedicated proxies, used either catchall SEOSpartans email or free yahoo/nokiamail
- created only article links (no profile or micromessage anchor text etc)
- all other settings on SER are identical
- HTML timeout - 180
- maximum size of website to download - 50MB
All dedi's have similar resources (around 8- 16GB ram, good processor),
dedi #1 has 1000mbps, dedi#2 and#3 have 100mbps uplink port speed.
All 3 dedi's are from popular hosting companies used widely for SER
All 3 dedi's run SER projects well with no major issues in getting verified links
RESULTS
- For this list, I get around 700 successfully created BuddyPress accounts created and successfully logged into
- Dedi #1 got around 100 verified article links - great result.
BUT,
Dedi #2 and Dedi #3 only got around 10-20 verified article links
FINDINGS
I choose to troubleshoot one URL - this URL I could consistently get a verified article posted with dedi#1, but never on dedi#2 or #3
- account creation, email verification and login where generally able to be created on all 3 dedi's, although occasionally account creation was not successful on dedi #2/3 (error: no form at all)
- article was created always on dedi #1, but never on dedi#2/3 (I tried disabling proxies, using different emails etc)
LOG
This is the most common log entry for failed article creation:
21:06:51: [+] 1/1 matches engine BuddyPress - http://domain.com/register/
21:06:51: [+] 1/1 new URL - http://domain.com/wp-login.php
21:06:51: [+] 1/1 login successful - http://domain.com
21:06:51: [+] 1/1 new URL - http://domain.com/groups/create/step/group-details/
21:06:51: [-] 1/1 no form at all - http://domain.com/groups/create/step/group-settings/
Even though SER did not create an article, I was able to manually log on using dedi #2/3 into the domain, and go to the page: http://domain.com/groups/create/step/group-settings/ and was able to manually post an article (using proxies or not using proxies)
@Sven,
I will PM you the URL to look at - I can't think of why I am getting a "no form at all" error, even though I have run this specific URL multiple times, and confirm that the domain is active and taking registrations.
I am happy to PM anyone else the URL if they also want to test
The reason I have gone into so much detail is to see whether there is any global issue with how SER is creating verified contextual links, and wonder whether this is an issue with the dedicated server or with SER.
I wonder whether there is an issue where SER is looking for an element on a page BEFORE the page has completely loaded. So if the element is not found, SER will log a "no form at all" and not try further with submission.
- it seems like SER might not be waiting long enough to get all page elements and so the submission fails
- SER only seems to try once to create an account and post article - if it fails during the submission, it does not seem to try again
- so even if a domain is down for a few seconds, the submission will fail
- Is there any way to try post an article again on accounts which have been successfully created? I know you can do scheduled article posting and post multiple times to a URL, but is there any way of adding option: "Set status ..> Active (log in and submit only)" ?
I'm going to continue troubleshooting to see the best way to get as many verified contextual links with SER as possible. The way I have my settings at the moment, I'm sure there are many links which SER should be able to post to but is not posting to.
SETTINGS
- Used Gscraper and custom footprints for BuddyPress - got around 6,000 unique domains to test with SER.
- ran the list on 3 different dedicated servers (stopped other projects while doing troubleshooting)
- 50 semi-dedicated proxies, used either catchall SEOSpartans email or free yahoo/nokiamail
- created only article links (no profile or micromessage anchor text etc)
- all other settings on SER are identical
- HTML timeout - 180
- maximum size of website to download - 50MB
All dedi's have similar resources (around 8- 16GB ram, good processor),
dedi #1 has 1000mbps, dedi#2 and#3 have 100mbps uplink port speed.
All 3 dedi's are from popular hosting companies used widely for SER
All 3 dedi's run SER projects well with no major issues in getting verified links
RESULTS
- For this list, I get around 700 successfully created BuddyPress accounts created and successfully logged into
- Dedi #1 got around 100 verified article links - great result.
BUT,
Dedi #2 and Dedi #3 only got around 10-20 verified article links
FINDINGS
I choose to troubleshoot one URL - this URL I could consistently get a verified article posted with dedi#1, but never on dedi#2 or #3
- account creation, email verification and login where generally able to be created on all 3 dedi's, although occasionally account creation was not successful on dedi #2/3 (error: no form at all)
- article was created always on dedi #1, but never on dedi#2/3 (I tried disabling proxies, using different emails etc)
LOG
This is the most common log entry for failed article creation:
21:06:51: [+] 1/1 matches engine BuddyPress - http://domain.com/register/
21:06:51: [+] 1/1 new URL - http://domain.com/wp-login.php
21:06:51: [+] 1/1 login successful - http://domain.com
21:06:51: [+] 1/1 new URL - http://domain.com/groups/create/step/group-details/
21:06:51: [-] 1/1 no form at all - http://domain.com/groups/create/step/group-settings/
Even though SER did not create an article, I was able to manually log on using dedi #2/3 into the domain, and go to the page: http://domain.com/groups/create/step/group-settings/ and was able to manually post an article (using proxies or not using proxies)
@Sven,
I will PM you the URL to look at - I can't think of why I am getting a "no form at all" error, even though I have run this specific URL multiple times, and confirm that the domain is active and taking registrations.
I am happy to PM anyone else the URL if they also want to test
The reason I have gone into so much detail is to see whether there is any global issue with how SER is creating verified contextual links, and wonder whether this is an issue with the dedicated server or with SER.
I wonder whether there is an issue where SER is looking for an element on a page BEFORE the page has completely loaded. So if the element is not found, SER will log a "no form at all" and not try further with submission.
- it seems like SER might not be waiting long enough to get all page elements and so the submission fails
- SER only seems to try once to create an account and post article - if it fails during the submission, it does not seem to try again
- so even if a domain is down for a few seconds, the submission will fail
- Is there any way to try post an article again on accounts which have been successfully created? I know you can do scheduled article posting and post multiple times to a URL, but is there any way of adding option: "Set status ..> Active (log in and submit only)" ?
I'm going to continue troubleshooting to see the best way to get as many verified contextual links with SER as possible. The way I have my settings at the moment, I'm sure there are many links which SER should be able to post to but is not posting to.
Comments
You can Export Account Data from the modify project option (right click the selected project). This gives a list of created accounts, together with username:password
thanks for taking the time to do some testing
There are 2 main difference with the settings you had on your project:
1. You have "allowed posting on same site again"
2. You have selected "all" on "Type of Backlinks to create"
This means that SER will continuously keep posting on the same site, which means that SER will have multiple chances to post to a particular domain. It also means that many of your links will be profile links rather than article links.
You've got 1199 unique domains, but only 7506 out of 28191 links are DoFollow, whereas BuddyPress article links should mostly be DoFollow ...> so I think if you check your verified links, you will see that many of them are not article links.
I did a small test run with your settings and this is what I got:
The settings that I was using on my previous testing is as follows:
I know that SER can create article links to BuddyPress articles, but I am doing troubleshooting to try to work out why many potential article links are NOT created with SER.
@satyr85, if you could do another test run with your BuddyPress list, but disable "posting on same site again", and create only "article" links, I would be very interested to see how many BuddyPress article links SER creates on a single run
And another thing, if you check "allowed posting on same site again" but you set maximum 1 article per site/domain from the article tab isnt' ser submitting just one article per site ? But yes, it will still try to submit to sites where it failed before, I'm just saying that ser will submit only one article per domain, right?
I think that if you have that option unchecked ser will:
- use the .hosts_done file and you will get the "already parsed" messages in the log
If you check that option ser will:
- use the .static file and try to register on websites that dont have accounts in the .static file
That's my opinion, i dont know for sure. Maybe someone more experienced can explain in detail the difference between check and not checking the "allowed posting on same site again" option, for contextuals.
I unselected the "allowed posting on same site again" option for troubleshooting, as I just wanted one article created per URL