Python Forum

Full Version: Does anyone know how to scrape/find this data?
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
No one seems to know how to do this. Maybe it's not possible.

I need to find out how quickly particular expired domains get registered after their release.

I know of two ways to do it but both are unviable:

Through the API of a registrar. Just make availability/registration calls but the API calls you get are way too few to be able to determine the speed.

Through paid whois requests: https://jsonwhois.com/pricing

but it's way too expensive. I'd need to spend something like hundreds of dollars per domain to do this. Absolutely non-practical.

If anyone has an alternative idea, I would appreciate it a lot.
You have not mentioned anything other that the pay site, no target site, Is this click bait?
(Jan-20-2019, 04:18 PM)Larz60+ Wrote: [ -> ]You have not mentioned anything other that the pay site, no target site, Is this click bait?

What do you mean? That's one of the two methods I know of. I am wondering if anyone knows another way to find this data.
No one online seems to know so maybe it's impossible.
so is json.whois the site you want to scrape?
(Jan-20-2019, 05:45 PM)Larz60+ Wrote: [ -> ]so is json.whois the site you want to scrape?

Nonono.

The site can be anything. I simply want to be able to scrape when any given domain (which I know will drop in a 3 hour window) is released, but I estimate it needs to be to at least 0.01 second accuracy. Maybe even 0.001 but I doubt it.

The site I would use for that IS the question. So json.whois is too expensive for the accuracy I'd need. Scraping through an API of a registrar is too slow because they can allocate you 1 to 10 calls per second (so 1 second or 0.1 second accuracy).

I don't know of a third method and it seems that no one does.
could you provide an example?
(Jan-20-2019, 05:53 PM)Larz60+ Wrote: [ -> ]could you provide an example?

Of a domain name?

pma.io

It will drop on 2019-01-21 01:45PST - 4:45 PST.

It will be caught quickly after it drops. I'm trying to find out how quickly. My estimate is about 0.01 seconds but the exact information is purposefully held back.
Ok, now I understand, I don't know how you would know without polling.
I guess it's impossible to find out in another way since no one knows the answer.

I've also tried finding out with a little bit of social engineering but no luck. The polling site I linked actually doesn't even allow that many calls. I'd need to make more accounts and spend $four digits to find out. Oh well.