Does/can/how gsa grab all commentable urls from a blog comment based platform?

googlealchemist

If I have one, or more, urls on a blog comment based platform, and/or just the root domain, that allows comment links to be dropped...does gsa already have this function, or could it be implemented, that it will grab all the other internal urls on that domain that also are open to comments?

Or do i have to check all the subbmitted/verified urls and sort by the blog comment related engines, and then do a site: or link extractor function in scrapebox, or whatever, to grab them all myself?

And before the Karen newbies chime in, im not worried about too many links or diminishing returns from the same domain blah blah blah.

Thanks

cherub

For blog comments, once I have a decent sized verified list, I extract all the domains and then scrape site:domain.com for each of them, adding in some stopwords to try and get around the limitations Google gives on those sorts of searches. This usually gives me a load of other blog comment urls from those domains.

Sven

There is a script command for doing this per engine, but got blog comments it is a general engine that would not work for specific ones. I need to look into this then.

googlealchemist

cherub said:

For blog comments, once I have a decent sized verified list, I extract all the domains and then scrape site:domain.com for each of them, adding in some stopwords to try and get around the limitations Google gives on those sorts of searches. This usually gives me a load of other blog comment urls from those domains.

thanks, thats what i figured my plan would be as well.
what do u mean about the stop words to get around google limitations?

googlealchemist

Sven said:

There is a script command for doing this per engine, but got blog comments it is a general engine that would not work for specific ones. I need to look into this then.

thatd be awesome if u could automate this in some way...

not sure what u mean about the specific vs general engines though, i see about 20 different specific types of blog comments under the blog comment section?

cherub

googlealchemist said:

cherub said:

For blog comments, once I have a decent sized verified list, I extract all the domains and then scrape site:domain.com for each of them, adding in some stopwords to try and get around the limitations Google gives on those sorts of searches. This usually gives me a load of other blog comment urls from those domains.

thanks, thats what i figured my plan would be as well.
what do u mean about the stop words to get around google limitations?

Google will only display around 300-400 results max for any search, no matter if they say they have thousands of results available. Adding simple keywords or stopwords, negative keywords etc will usually bring back a slightly different set of results with possibly new urls not previously given.

googlealchemist

cherub said:

googlealchemist said:

cherub said:

For blog comments, once I have a decent sized verified list, I extract all the domains and then scrape site:domain.com for each of them, adding in some stopwords to try and get around the limitations Google gives on those sorts of searches. This usually gives me a load of other blog comment urls from those domains.

thanks, thats what i figured my plan would be as well.
what do u mean about the stop words to get around google limitations?

Google will only display around 300-400 results max for any search, no matter if they say they have thousands of results available. Adding simple keywords or stopwords, negative keywords etc will usually bring back a slightly different set of results with possibly new urls not previously given

never realized they throttled site: searches to grab all the inner urls, as long as i had enough proxies to avoid the issue there, thanks for the tip!