Error after a few minutes of running - julio2000 - Feb-05-2020
Error: Fatal Python error: Cannot recover from stack overflow.
Python runtime state: initialized
Current thread 0x000019e8 (most recent call first):
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\utils.py", line 57 in _has_surrogates
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\_policybase.py", line 287 in _sanitize_header
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\_policybase.py", line 316 in header_fetch_parse
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\message.py", line 471 in get
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\message.py", line 578 in get_content_type
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\message.py", line 594 in get_content_maintype
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\feedparser.py", line 295 in _parsegen
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\feedparser.py", line 180 in _call_parse
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\feedparser.py", line 176 in feed
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\parser.py", line 56 in parse
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\email\parser.py", line 67 in parsestr
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\http\client.py", line 221 in parse_headers
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\http\client.py", line 327 in begin
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\http\client.py", line 1322 in getresponse
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 416 in _make_request
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 665 in urlopen
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\poolmanager.py", line 330 in urlopen
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\request.py", line 171 in request_encode_body
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\request.py", line 79 in request
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 397 in _request
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 374 in execute
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 319 in execute
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976 in find_element
File "C:\Users\Julius\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 598 in find_element_by_css_selector
File "C:\Users\Julius\Desktop\test\Klad2.py", line 51 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
File "C:\Users\Julius\Desktop\test\Klad2.py", line 58 in trycart
So I've created this piece of code where it monitors the website of https://www.Jdsports.nl. My script refreshes the website and checks if the sneakers are in stock, it repeats this for as long as possible. As I said, this script works perfectly until it has refreshed the site for about 1500 times (if my maths are correct :)). After 1500 times it gives the error above^^^^. I'm running my code with headless selenium. Does anyone know why I get that error??
RE: Error after a few minutes of running - Gribouillis - Feb-05-2020
The error traceback shows a high level of recursion. Try to break the recursivity of the algorithm.
RE: Error after a few minutes of running - micseydel - Feb-05-2020
I suspect you want this (it's probably not worth changing your algorithm). If you do need to change your algorithm, I suspect you'll need to manage a stack manually.
RE: Error after a few minutes of running - julio2000 - Feb-05-2020
(Feb-05-2020, 08:18 PM)micseydel Wrote: The error traceback shows a high level of recursion. Try to break the recursivity of the algorithm. I tried to add the line:
sys.setrecursionlimit(10000) But nothing really changed :O.
(Feb-05-2020, 07:52 PM)Gribouillis Wrote: The error traceback shows a high level of recursion. Try to break the recursivity of the algorithm. I'm sorry I'm a little bit of a noob but I don't understand what you mean and how to solve it oeps :O
RE: Error after a few minutes of running - Gribouillis - Feb-05-2020
julio2000 Wrote:My script refreshes the website and checks if the sneakers are in stock, it repeats this for as long as possible This task does not normally need a high level of recursion. It means that you are repeating the task by the means of recursion instead of repeating it by an infinite loop. The structure of the program should be
while True:
refresh_the_website()
check_if_the_sneakers_are_in_stock() None of these functions should call itself, directly or indirectly.
RE: Error after a few minutes of running - micseydel - Feb-05-2020
(Feb-05-2020, 09:35 PM)Gribouillis Wrote: The structure of the program should be If they're performing a depth-first search, then recursion is a very natural solution, and a simple loop won't be enough without them managing a stack manually, which isn't necessarily worth doing...
That said, it sounds like there's no base-case. Usually for a recursive web crawler like this, you'd specify a max depth.
You should probably share your code if you expect more help.
RE: Error after a few minutes of running - julio2000 - Feb-05-2020
(Feb-05-2020, 09:35 PM)Gribouillis Wrote: julio2000 Wrote:My script refreshes the website and checks if the sneakers are in stock, it repeats this for as long as possible This task does not normally need a high level of recursion. It means that you are repeating the task by the means of recursion instead of repeating it by an infinite loop. The structure of the program should be
while True:
refresh_the_website()
check_if_the_sneakers_are_in_stock() None of these functions should call itself, directly or indirectly. and how do I say when it needs to be False?
RE: Error after a few minutes of running - Gribouillis - Feb-05-2020
As @micseydel said, we know too little to give a valuable help.
From the traceback, we can see that the error occured during the call of a function named find_element_by_css_selector() . Again I cannot imagine such a function involving a high level of recursion without an error in the logic of the program. Typically, a function with that name will traverse a document's tree which depth is small.
RE: Error after a few minutes of running - julio2000 - Feb-05-2020
(Feb-05-2020, 09:51 PM)Gribouillis Wrote: As @micseydel said, we know too little to give a valuable help.
From the traceback, we can see that the error occured during the call of a function named find_element_by_css_selector() . Again I cannot imagine such a function involving a high level of recursion without an error in the logic of the program. Typically, a function with that name will traverse a document's tree which depth is small. my script is kind of as following:
def Trycart():
try:
check if it's instock
except Exception: <- when it's out of stock it will give an Exception
Trycart <- so when it's out of stock it will try this function again It is a bit more complicated then this^^ but this kind of sums it up. So I'm not using a while loop but I'm constantly repeating the function. Could this b the problem?
RE: Error after a few minutes of running - Gribouillis - Feb-05-2020
Yes I do think it could be the problem. You could replace this for example with
while True:
try:
check if it's in stock
except Exception:
continue
else:
break
|