Python Forum
Does anyone know how to scrape/find this data?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Does anyone know how to scrape/find this data?
#1
No one seems to know how to do this. Maybe it's not possible.

I need to find out how quickly particular expired domains get registered after their release.

I know of two ways to do it but both are unviable:

Through the API of a registrar. Just make availability/registration calls but the API calls you get are way too few to be able to determine the speed.

Through paid whois requests: https://jsonwhois.com/pricing

but it's way too expensive. I'd need to spend something like hundreds of dollars per domain to do this. Absolutely non-practical.

If anyone has an alternative idea, I would appreciate it a lot.
Reply
#2
You have not mentioned anything other that the pay site, no target site, Is this click bait?
Reply
#3
(Jan-20-2019, 04:18 PM)Larz60+ Wrote: You have not mentioned anything other that the pay site, no target site, Is this click bait?

What do you mean? That's one of the two methods I know of. I am wondering if anyone knows another way to find this data.
No one online seems to know so maybe it's impossible.
Reply
#4
so is json.whois the site you want to scrape?
Reply
#5
(Jan-20-2019, 05:45 PM)Larz60+ Wrote: so is json.whois the site you want to scrape?

Nonono.

The site can be anything. I simply want to be able to scrape when any given domain (which I know will drop in a 3 hour window) is released, but I estimate it needs to be to at least 0.01 second accuracy. Maybe even 0.001 but I doubt it.

The site I would use for that IS the question. So json.whois is too expensive for the accuracy I'd need. Scraping through an API of a registrar is too slow because they can allocate you 1 to 10 calls per second (so 1 second or 0.1 second accuracy).

I don't know of a third method and it seems that no one does.
Reply
#6
could you provide an example?
Reply
#7
(Jan-20-2019, 05:53 PM)Larz60+ Wrote: could you provide an example?

Of a domain name?

pma.io

It will drop on 2019-01-21 01:45PST - 4:45 PST.

It will be caught quickly after it drops. I'm trying to find out how quickly. My estimate is about 0.01 seconds but the exact information is purposefully held back.
Reply
#8
Ok, now I understand, I don't know how you would know without polling.
Reply
#9
I guess it's impossible to find out in another way since no one knows the answer.

I've also tried finding out with a little bit of social engineering but no luck. The polling site I linked actually doesn't even allow that many calls. I'd need to make more accounts and spend $four digits to find out. Oh well.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
Question in this code, I input Key_word, it can not find although all data was exact Help me! duchien04x4 3 971 Aug-31-2023, 05:36 PM
Last Post: deanhystad
  get data from excel and find max/min Timmy94 1 1,077 Jul-27-2022, 08:23 AM
Last Post: Larz60+
  what will be the best way to find data in txt file? korenron 2 1,126 Jul-25-2022, 10:03 AM
Last Post: korenron
  Find string between two substrings, in a stream of data xbit 1 2,113 May-09-2021, 03:32 PM
Last Post: bowlofred
  Find data using a period of time in SQLITE3 SmukasPlays 2 2,142 Jul-30-2020, 02:02 PM
Last Post: SmukasPlays
  Find offset of binary data in file sparkz_alot 3 5,848 Mar-01-2017, 10:30 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020