This is will not work. While the php code is fine there is one major issue which makes doing this a waste of bandwith and will actually not give any benefit. I will try to explain:
This will not work as the domain links you submit have to have your specific Index Now API .txt file hosted on them as well. Hence as the owner of lets say sitea.com, I cannot directly submit links from siteb.com or sitec.com etc via IndexNow. They took this obvious measure to prevent being mass spammed. Its common sense really.
Therefore you need to do one of the following:
1.) Either create your own link on your own domain hosting the above mentioned file that 301 redirects to the url and submit the redirect link with IndexNow. I recommend this.
or
2.) Host a link portal on your page (with your created links) and submit that page of that link portal, hoping the spiders follow all the external links. This option is risky and I do not recommend this. Link farms are already filtered from 20 years ago and this is likely to do no benefit.
This makes sense, thanks for explaining in detail.
I've tried it with some URLs of my money sites for which I have created YOURLs URLs on another server (with the indexnowapi.txt file on it).
Submission works fine again, but no crawler is showing up when I look into the Apache log files. Not sure if there is a huge delay between submission and the bot showing up but there was none in the past 1.5 hours.
This is will not work. While the php code is fine there is one major issue which makes doing this a waste of bandwith and will actually not give any benefit. I will try to explain:
This will not work as the domain links you submit have to have your specific Index Now API .txt file hosted on them as well. Hence as the owner of lets say sitea.com, I cannot directly submit links from siteb.com or sitec.com etc via IndexNow. They took this obvious measure to prevent being mass spammed. Its common sense really.
Therefore you need to do one of the following:
1.) Either create your own link on your own domain hosting the above mentioned file that 301 redirects to the url and submit the redirect link with IndexNow. I recommend this.
or
2.) Host a link portal on your page (with your created links) and submit that page of that link portal, hoping the spiders follow all the external links. This option is risky and I do not recommend this. Link farms are already filtered from 20 years ago and this is likely to do no benefit.
This makes sense, thanks for explaining in detail.
I've tried it with some URLs of my money sites for which I have created YOURLs URLs on another server (with the indexnowapi.txt file on it).
Submission works fine again, but no crawler is showing up when I look into the Apache log files. Not sure if there is a huge delay between submission and the bot showing up but there was none in the past 1.5 hours.
Bing and Yandex usually show up within minutes. Then usually the next day other spiders start to do the rounds. Are you sure robots.txt is not blocking them?
Bing and Yandex usually show up within minutes. Then usually the next day other spiders start to do the rounds. Are you sure robots.txt is not blocking them?
@Momo well, this is my robots.txt below and crawling may be several thousand a day and certainly Google is nowhere to be seen (maybe a few).
He cant as he is trying to directly submit links he does not host. This will not work. Trust me, I tried this for a while and checked results before I went through the effort of building my own solution. I didn't do it just to waste my time lol.
Can you send me in private message the code for your solution ?
No. I deleted all the python package I created for the public as people were abusing it by selling services to the public for something that is free. Deleted it 15th July, around 2 days after you joined the forum. Unfortunately, you missed it.
Notice the the newcomers: I am NOT sharing my scripts with people who I do not know. I am not accusing anyone on particular, but there are several people who have join the forum with new accounts and then message me multiple times in their first 24 hours asking for my source code saying I can trust them. Sorry but not sorry.
He cant as he is trying to directly submit links he does not host. This will not work. Trust me, I tried this for a while and checked results before I went through the effort of building my own solution. I didn't do it just to waste my time lol.
Can you send me in private message the code for your solution ?
No. I deleted all the python package I created for the public as people were abusing it by selling services to the public for something that is free. Deleted it 15th July, around 2 days after you joined the forum. Unfortunately, you missed it.
Notice the the newcomers: I am NOT sharing my scripts with people who I do not know. I am not accusing anyone on particular, but there are several people who have join the forum with new accounts and then message me multiple times in their first 24 hours asking for my source code saying I can trust them. Sorry but not sorry.
Hi mate I've messaged you about this, and I'm not a newcomer seasoned people like @sickseo can vouch for me.
Comments
I've tried it with some URLs of my money sites for which I have created YOURLs URLs on another server (with the indexnowapi.txt file on it).
Submission works fine again, but no crawler is showing up when I look into the Apache log files. Not sure if there is a huge delay between submission and the bot showing up but there was none in the past 1.5 hours.
well, this is my robots.txt below and crawling may be several thousand a day and certainly Google is nowhere to be seen (maybe a few).
Notice the the newcomers: I am NOT sharing my scripts with people who I do not know. I am not accusing anyone on particular, but there are several people who have join the forum with new accounts and then message me multiple times in their first 24 hours asking for my source code saying I can trust them. Sorry but not sorry.