Big scrape domain (by extension) reward

Kaine

Hi everyone. 

I just offer you something interesting and only participants will be rewarded. 

I created lists for bruteforce domaine scraping. It's lists are sorted by extension. 

Each list contains 475,255 rows Google gives 1000 results by search ...

each list are going to do more than 100 million results in minimum (domain dedupe trim to root).

Each person who scrape a list have the right to retrieve the result of others. 

The scrape must be complete and realized on google without footprint, maximum result per page = 1000. The goal is to retrieve a maximum domain. 

If someone is interested in. which manifests and say what he want to deal list. 

Do not drop matches on the forum, the exchange will be made by MP (so that those who want to enjoy, do them).

Here are the lists available at the moment, if you want a particular called me.

Kaine

I start .fr list.

DonCorleone

i didn't understand what's in those files .

Kaine

I send you pm

Kaine

If you are interested ask for the list to scrape that interests you.

jjumpm2

@Kaine I will scrape .co.uk

Kaine

Ok i pm you

fakenickahl

I'd do the .com

Kaine

Hi fakenickakl

.com is specifique (multi-langage) specify langage used after scraping.

I PM you

At this time i have + 1 000 000 domain Fr

DonCorleone

Okay i will take le german.

Kaine

Thank's PM sent too

Kaine

.Net just taken too.

These are not lists of words, but request. The interest is to recover a maximum of domain extensions. Google is the best database.

Summary file at this time and still available:

(Once the work is finished it will not be possible to recover the joint work)

If you see another interesting extension to add mention it.

ewandy

I will scrape .org

Kaine

I PM you

.Org taken too.

gooner

@kaine - I'll do it too, give me without extension you want, i don't mind

Kaine

lol I don't know ... maybe .edu or .gov may be interested people

gooner

Sure, i'll do .edu

Kaine

Ok i PM you

gooner

Thanks

Kaine

It is I who thank you. 

I think we'll have a big packet domain to exchange

.Edu is taken now.

History to identify the thing: It's free labor.

 

Each manages it as he wants, all the participants are free to exchange their list when they want. 

The only rule is to share only to those who will participate. 

If someone wants to stop, he should say to liberate space for another member. 

I think we'll have to sort all lists in one place so that participants can access a time the job ends.

Trevor_Bandura

What ones are still available?

I'll do .org I guess if thats still available.

Kaine

Same at last picture less:

-.Org

-.Edu

Trevor_Bandura

.ca available?

Kaine

Yes I PM you

.Ca taken now.

MAJ

Kaine

.Ch taken too now.

vort3x

I'll do the .ch

Kaine

I have Pm you

Kaine

For those who have started their list, how it goes? (Normally even in dedoublonnant on the fly, the file must grow very quickly).

Once completed I need to know if multi-language extension could benefit from complementary scrape. 

Change Language, test a little and see if the domains are added.

Do not forget to trim at root and deduplicated, otherwise the files will be huge

I advise those who are not yet working with us to choose a list quickly. Take any Laquel, you will get the work of others. 

Once finished it will be impossible to retrieve this huge list of domains.

We will not share this work.

vifa

I want to participate. Do you have any list left?

Kaine

Yes you have préférence ?

Lest screen less: .Ch

I send you by PM: .gov

.Gov now taken.

Actual refreshed list to scrap:

If you do not add new domain to scrap it remains only 9 seats available.

jjumpm2  is responsible for domain.CO.UK

fakenickahl  is responsible for domain.COM

DonCorleone is responsible for domain.DE

Justin is responsible for domain.NET

ewandy is responsible for domain.ORG

gooner is responsible for domain.EDU

Trevor_Bandura is responsible for domain.CA

vort3x is responsible for domain.CH

vifa is responsible for domain.GOV

@kaine is responsible for domain.FR

DonCorleone

sorry , I will be out of town till Monday . If everyone is in a hurry , please scrape .de domains . I had scraped 5K domains till now . .de File is on my wall . Otherwise I will resume scraping after monday . sorry again .