How to get more Web2.0 verified ? (+ Wikidot Script inside)

Crapper

Hi,

I'm useing SER for a week or 3 now.(quite happy) But I've noticed that there are not many Web2.0's verified. 

I'm updating everyday to get the last version and saw that there are many Web2.0's to choose from. (I've selected all)

Also I've allready tried to create a project with only Web2.0's selected. The result of a week was only 1 veryfied.

For keywords I'm using about 10.000 diffrent ones, so that should not have been the problem.

Am I doing something wrong ?

Thanks to anyone helping me out !

m3ownz

What do the logs say?

Make a dummy project just for web2.0s, set the thread count low and set debug mode ON under advanced options. Then start the project and read the logs, they should be pretty clear about what is or isnt working.

For example, if you are using CS and most of the web2.0s use recaptcha, you wont get many links. 

Crapper

Ok,

I do use CS so that maybe the problem because I've also checked to don't fill captcha manually.

So how low does the thread count need to be ? 10 ?

Thanks for your help

Ozz

That won't help as most sites use recaptcha which isn't supported by CS. You need a 2nd captcha solving service or register to ShaniBPO. They give you 100 captchas for free and if you are lucky another 500 for testing.

m3ownz

The low thread count is just to make it easier to debug, so the log doesnt fill up at 100 entries a second, there is no set count, but 10 would probably be fine. If im debugging in detail i will set it to 1 thread. 

If you are only using CS then i expect that is the issue and you can probably skip debugging and try as Ozz suggests and use a backup service, i use DBC.

Crapper

Thanks for the help guys !!! Already have 10 verified web2.0's :-)

I've started this project but than I noticed that it still didn't do very much. Than I removed the history and cache.

Thanks for the tip Ozz. Going to try ShaniBPO soon. Getting tiered of filling captcha's..... :-)

Crapper

Ok,

I've got 19 verified web2.0

But is it normal that after verifying some that SER stops for a while ?

It's not doing anything for a hour or 2

Ozz

You can only post to them once per email. The only exeption is  n****.l* or Web 2.0's where you don't need an email to register.

Crapper

Aha I see.... 

I didn't realized that. So when I want more I have to create a new engine for that, right ?

Ozz

Yes (in general). You can change your email though if you want to submit to these sites twice. Just imagine that you register to youtube. That is a Web 2.0 site where you register once with one email only. You aren't allowed to register with the same email and a different user name again.

There are some exceptions like " n****.l*" which built upon the same framework. Another example would be "Sun Media". If you want to script this engine you can use the the footprint " "are members of Canoe Sun Media Community Newspapers." ". But its not as easy as you might hoping

Crapper

I saw a post(tutorial) on creating an engine for web2.0 So I will thake a look at it now. Doesn't seem to be very complicated, but I haven'd tried it yet...:-)

What's strange though, is that I only get 19 veryfied and there are 36 engines for web2.0 right now

What could be the reason for that ?

Ozz

Give it a try . But remember that every site is different. Post your script in "New Engines/Platforms" and I try to help out as good as I can. If nothing helps than Sven has to fix it

I try to complete the tutorial today or tomorrow. A good advice is to compare some Web20 scripts with each other and to register to some of these sites by hand to get a good understanding for this.

To your question: This has to be answered by Sven. I think I know the reason (captcha wasn't solved and site is marked as "already parsed"), but I don't want to write some bullshit about this.

Crapper

I'll give it a shot,soon :-)

Can you please let me know when your tutorial is finished ?

I gues I need a captcha service then :-)

Thanks for your help

Ozz

you can use the tutorial right now. it is basically ready to use and I just have to add some write up about extra-steps to take and how to find a form variable for %article%. you should also read the phpfox-tutorial which is mentioned in that thread.

do you have any scripting/programming/web-developing background? you don't have to but it might come in handy to understand some things. i for myself don't had any clue about this and had to learn everything. almost every script of mine had to be fixed by Sven, too. 

i give my best, but the learning curve is very slow for me, hehe

Bytefaker

If you don't have a programmer background Ozz, you're a damn fast learner!

m3ownz

Im not sure a programming background is all that helpful. I spend forever trying to figure out the underlying programming logic behind every switch, where as Ozz seemed to have just jumped right in at the deep end.

Whos approach worked out the best? Well how many engines do you see with my name on them?

Bytefaker

I know this from myself... So Ozz impresses me every time again. He has a good feeling for the right stuff and never give up. And his instructions are really nice with screenshot - such efforts are just damn great and let our little community grow

PS: I don't think you have to be a programmer, but you have to know how basic stuff like GET/POST works - then and with this instructions I think it should be possible for all really interested users.

Ozz

thanks to all!

what helped me obv was to create the "GSA - Scripting" manual for notepad++. i just put that in my second view of notepad++ and compare some engines with that manual. if I'm not sure about a variable i just search for that term in that manual.

Ozz

i think what is really necessary to understand is how to find the right terms in a html source code and how every step is working.

in general i would say what you would do yourself when register to a site, GSA has to do it the same way in the same order.

before anyone register to a site and post a link you have to deliberate what

- Username you want to choose

- Password you want to use

- What link I like to post

- What text I like to post

those things are managed in the [setup].

once you know these variables you want to register. if the site has one fixed url to register and you know of this than you would copy/paste it into your browser or click your bookmark.

GSA does the same thing with "modify url=". 

if you don't know the exact register-url you would search and click the register-link on that site first.

GSA does the same thing with "find url=URLofLINK" and "find link=NAMEofLINK". Once it has find the correct link GSA is going to click that.

The steps GSA does take in general are:

[setup] -> define all variables

[register] -> what you would click to register to a site. if thats a multi-step-registration than [register_step2]...

[login] -> GSA has waited half a day for validation email and log in to the site with an approved account

[step] -> once you are logged in you click all links you need to post your link. every site is one step again [Step2]...

[extra-step] ->if you are able to post a second backlink on that site you have to make use of extra step(s)

Crapper

Well I really like the idea of sharing this and I will look at this tonight (I'm Dutch ny the way)

I've got some experiance as a php developer so GET/POST are not a problem at all. :-)

Let get this roll'n because this software rocks !!!

Crapper

I've been following your tut, Ozz, but I have some questions.

When you go to the login step, you first logout, right ?

submit success=>Logout<

submit failed=Sorry, but the login information you provided was incorrect.

but where does your submit failed come from ?

Also I've used the template and is it correct that i need to remove all variables that are not needed ?

Thanks

Ozz

I've been following your tut, Ozz, but I have some questions.

When you go to the login step, you first logout, right ?

Yes. In general my process to script is to "Register -> Logout -> Login -> Move to Preferences/Profile/Blog"

submit success=>Logout<

submit failed=Sorry, but the login information you provided was incorrect.

but where does your submit failed come from ?

A simple way to find that failed messages for register and log in is to click the Register/Login-Button with empty fields. Another way is to log in with false passwords, usernames, wrong captcha code, etc.

On some sites you only get a temporary popup. If thats the case than I leave "submit failed=" empty or disregard the line with ";".

>Logout<, >Sign out<, etc. are easy to find in the html source once you have logged in to a site. Just find the text of the "Logout"-link of that page and search with the source for that.

Also I've used the template and is it correct that i need to remove all variables that are not needed ?

Yes, variables like [Website_title] or [About_Yourself] which aren't needed can be removed from the script.

The template is built to cover most of the common Web 2.0's. For some sites you need to define variables, for most you don't need every variable.

Crapper

Thanks for your anwser Ozz.
This is what I've got so far. Can you please check if it's correct ? I know that the article submission is not there yet, but just want to check if I'm doing this correct....

[setup]
enabled=1
default checked=0

fixed url=https://www.wikidot.com/default--flow/login__CreateAccountScreen
url must have=*://*www.wikidot.com*

engine type=wikidot
description=Creates a blog entry on www.wikidot.com

dofollow=1
anchor text=1
uses pages=0

extract keywords=0

skip ext links on=0
skip content on=0
skip url content on=0
posted domain check=1
       
[Your E-Mail]
type=email

[URL]
type=url

[Anchor_Text]
type=text

[About_Yourself]
type=memo
allow html=1
html to bbs=1
allow return=1
must be filled=0
hint=Please fill this with some information about yourself. Use the spin syntax heavily here.
custom mode=1
auto modify=2
default=%file-about_yourself.dat%
auto add anchor url=1
;auto add anchor url content={{Check out|Stop by|Visit|Review|Take a look at|Have a look at|Look at|Look into} my|My|my|Here is my|Also visit my|Feel free to {visit|surf to} my} {web page|web site|website|webpage|site|page|blog|homepage|web-site|weblog|web blog|blog post}{:| ::|;| -| ...|} <a href="%url%">%anchor_text%</a> - %url%

[Article_Title]
type=text
allow html=0
must be filled=1
hint=capitalize each word.
max length=45

[Article]
type=memo
allow html=1
html to bbs=1
must be filled=1
hint=The full article comes here.
auto modify=1


[Login]
type=login
must be filled=1
hint=The login for websites that need an account. Use numbers and letters only.
min length=5
static=1
upcase=0

[Password]
type=password
must be filled=1
hint=A password used for websites that need an account. Use numbers and letters only.
min length=6
static=1

[Website_title]
type=text
allow html=0
must be filled=1
hint=Thats usually a anchor text, so please make sure it holds an important keyword.

[Description_450]
type=memo
max length=450
allow html=1
must be filled=1
hint=Use spin syntax here to not get duplicate content.
auto add anchor url=2

[verify_url]
type=extract
find link=
find url=

;*/read_blog/[0-9]*[0-9]/*[a-z]*

;front1=/read_blog/
;front2=
;back1="
;back2=

[REGISTER_STEP1]
modify url=https://www.wikidot.com/default--flow/login__CreateAccountScreen

submit success=We have just sent a confirmation email to you. Please check your email box and click on a link.
submit failed=Please provide the name you want to use|Please provide a valid email address|Your password must be at least six characters: please try again with a longer and more secure password|Please re-enter the verification code
submit success skip verify=This username already exists.|This email is already being used.

form name=*button|*Get me a free account!
form url=https://www.wikidot.com/default--flow/login__CreateAccountScreen
form id=createaccount-form0

verify submission=1
verify interval=10
verify timeout=30
first verify=0
verify by=email
verify on unknown status=1

set unknown variable=%leave%


email=%your e-mail%
password=%password%
password2=%password%
recaptcha_response_field=%captcha%
name=%spinfile-names.dat%
language=%leave%
verify by=email+search,url
verify search for=%website_title%,%url%

[LOGIN_STEP1]
modify url=https://www.wikidot.com/default--flow/login__LoginPopupScreen?originSiteId=1&openerUri=http://www.wikidot.com

submit success=Sign out
submit failed=Sorry, but the login information you provided was incorrect.

verify on unknown status=1

form name=*login-form|*Sign in
form url=https://www.wikidot.com/default--flow/login__LoginPopupScreen?originSiteId=1&openerUri=http://www.wikidot.com


set unknown variable=%leave%

login=%your e-mail%
password=%password%


[STEP1]
modify url=http://www.wikidot.com/account/settings#/about

form name=*Apply
form url=http://www.wikidot.com/account/settings#/about
form id=dp-about-form

set unknown variable=%leave%

real_name=%spinfile-names.dat%
gender=%random_option%
about=%about_yourself%
website=%url%
im_aim=%leave%
im_gadu_gadu=%leave%
im_google_talk=%leave%
im_icq=%leave%
im_jabber=%leave%
im_msn=%leave%
im_yahoo=%leave%
location=%leave%

Ozz

lol, believe it or not, I just tried to script that site. It's very fresh and untested by now.

I will move this thread to "New Platforms" and post my script to compare both. Mine will have some errors and yours propably too. But my first impression is that you've done a very good job. Impressive!

Ozz

;extra-step2 is javashit

[setup]

enabled=1

default checked=0

fixed url=https://www.wikidot.com/default--flow/login__CreateAccountScreen?originSiteId=1

url must have=*://www.wikidot.com*

engine type=My Web 2.0

description=Creates a blog entry on www.wikidot.com

dofollow=1

;not defined atm

anchor text=1

uses pages=0

extract keywords=0

skip ext links on=0

skip content on=0

skip url content on=0

posted domain check=1

        

[Your E-Mail]

type=email

[URL]

type=url

[Anchor_Text]

type=text

[About_Yourself]

type=memo

allow html=1

html to bbs=1

allow return=1

must be filled=1

hint=Please fill this with some information about yourself. Use the spin syntax heavily here.

auto modify=0

default=%file-about_yourself.dat%

[Article_Title]

type=text

allow html=0

must be filled=1

hint=capitalize each word.

max length=45

[Article]

type=memo

allow html=1

html to bbs=1

must be filled=1

hint=The full article comes here.

auto modify=1

[Login]

type=login

must be filled=1

hint=The login for websites that need an account. Use numbers and letters only.

min length=5

static=1

upcase=0

[Password]

type=password

must be filled=1

hint=A password used for websites that need an account. Use numbers and letters only.

min length=6

static=1

[Website_title]

type=text

allow html=0

must be filled=1

hint=Thats usually a anchor text, so please make sure it holds an important keyword.

[REGISTER_STEP1]

modify url=https://www.wikidot.com/default--flow/login__CreateAccountScreen?originSiteId=1

submit success=We have just sent a confirmation email to you. Please check your email box and click on a link.|Please check your emailbox

submit failed=id="ca-reg0-errors"|Please re-enter the verification code|Please provide the name you want to use|Please provide a valid email address|You did not correctly confirm your password:|

submit success skip verify=

form name=*Get me a free account!

form url=https://www.wikidot.com/default--flow/login__CreateAccountScreen?originSiteId=1

form id=createaccount-form0

verify submission=1

verify interval=10

verify timeout=30

first verify=0

verify by=email

verify on unknown status=1

set unknown variable=%leave%

name=%login%

email=%your e-mail%

language=en

password=%password%

password2=%password%

[LOGIN_STEP1]

modify url=https://www.wikidot.com/default--flow/login__LoginPopupScreen?originSiteId=1&openerUri=http://www.wikidot.com

submit success=href="http://www.wikidot.com/user:info/%login%">|>Sign out<

submit failed=The login and password do not match.

verify on unknown status=1

form name=*Sign in

form url=https://www.wikidot.com/default--flow/login__LoginPopupScreen?originSiteId=1&openerUri=http://www.wikidot.com

set unknown variable=%leave%

login=%login%

password=%password%

bind_to_ip=%leave%

[STEP1]

modify url=http://www.wikidot.com/account/settings#/about

;submit success=

;submit failed=

form name=*Apply

;<a class="awesome " onclick="jQuery(this).parents('form').submit();" href="javascript:;">Apply</a>

form url=http://www.wikidot.com/account/settings#/about

form id=dp-about-form

verify submission=1

verify interval=1

verify timeout=1

verify url=http://www.wikidot.com/user:info/%login%

first verify=0

verify by=url

verify on unknown status=1

set unknown variable=%leave%

variable must be used=url

real_name=%spinfile-names.dat% %spinfile-lnames.dat%

gender=%random_option%

birthday_day=%random_option%

birthday_month=%random_option%

birthday_year=%random-1965-1990%

about=%about_yourself%

website=%url%

im_aim=%leave%

im_gadu_gadu=%leave%

im_google_talk=%leave%

im_icq=%leave%

im_jabber=%leave%

im_msn=%leave%

im_yahoo=%leave%

location=%columnspinfile-address_data.dat-3%, %columnspinfile-address_data.dat-1%

[EXTRA_STEP1]

modify url=http://www.wikidot.com/account/sites#/new

submit success=%article_title%%

;submit failed=

form name=*Get my Wikidot site

;<input type="button" onclick="WIKIDOT.modules.NewSiteModule.listeners.next(event)" value="Get my Wikidot site">

form url=http://www.wikidot.com/account/sites#/new

form id=new-site-form

verify submission=1

verify interval=1

verify timeout=1

first verify=0

verify by=url

verify url=http://%login%.wikidot.com/blog:_start

verify on unknown status=1

set unknown variable=%leave%

name=%website_title%

subtitle=%keyword%

unixname=%login%

language=en

template=blog-template

privacy=closed

;closed — anyone can view, but membership is restricted

tos=1

[EXTRA_STEP2]

modify url=http://%login%.wikidot.com/

form name=*save

form url=http://%login%.wikidot.com/

form id=edit-page-form

set unknown variable=%leave%

variable must be used=url

title=%article_title%

source=%article%

comments=%leave%

Crapper

Hahaha....LOL

That's funny !!!

Thanks

So how are we testing this ?

Ozz

Debugging is a pain in the ass. Especially for Web 2.0's because you only have one URL to post to. If your registration process works and you have a problem within the login- or posting-steps than you have burned your email for this.

I have created up to 10  new projects for testing with different emails of course. 

Best practice to test your script you have to stop all projects, copy your "wikidot.ini" into the "/engines" folder and activate the engine in the project. After you have done that activate the debugger (options -> advanced -> run in debug mode) and start only the test project. You have to notice everything in the debugger and log window. 

If the log says (captcha couldn't be solved or something like that), than stop the project and "delete url history" and start again (with changes if needed).

If nothing helps and this is often the case, than send your script to Sven and he will fix everything . I feel always sorry for him, but sometimes you have no other choice.

To the script:

- "name=%login% or %spinfile%? spinfile is fine as you can log in with your email. 

- "About Yourself" -> I choosed the short version because the URL will be posted in an extra field. Need to change that in the template though.

- "description_450" and "verify_url" ar not needed this time. verify_url is easy to define this time (GSA don't need to find any link to your post to extract) so that you can "verfify url=" in the verification code of your script.

- ReCaptcha field don't need to be defined. GSA knows to handle them by default

- email verification is also automatically by default. I've never used "verify by=email+search,url" but maybe its a good idea, hehe. But in general GSA knows how to handle emails and click the right link

In conclusion I have to say that I couldn't have done it any better. Thumbs up!

Crapper

Ok,

Nice to hear that my script was good :-)
Do I need to restart GSA after I've made a change in the code ?

Would it be a good idea to split the work. Let's say I provide the basic code and someone else will debug. Or the other way around ? This way we will not have duplicate work as we had today and will speed things up. Todays duplicate work was good for my learning curve dough....

Crapper

I'm getting this error: unwanted capcha found

Allready removed the ReCaptcha field. But how to solve this ?
Also I've noticed that GSA is going on with searching. This is not needed for WEB2.0 I think so how do I stop that ?

Crapper

Just a tought here....

What if we duplicate a script and leave the register part out of it.
If we open up GSA and create a new project and then first use the first script with the login and password filled in. (not randomize)
After that we run the second script with the same login and password and we can post a second post to that particular website. This way we have a more than one post and would be better I think.

Would this work ?

Ozz

sorry, I don't have the chance to test this atm. 

"unwanted captcha" - I have no idea how to solve that  or what it mean yet. 

your thought about "to leave the register"-part could work out. I dunno, this has never comes to my mind, hehe. I just duplicated a project with a different email to do another test round. 

but i don't think that it is necessary to remove the registration part. just add "submit success skip verify=" to the script with the message that pops up when you have registerd with an approved email already.