Thank you both for answers.
@HarleyQuin
The code I ran months ago was same as I posted here, but result was not same. As stated, on my first attempt I got all the HTML contents, while this time I didn't. Also, replacing the parser for lxml parser didn't make a difference. Do you have any idea, from experience, why such difference?
@snippsat
Your code returns all the HTML contents of the page, if I print the soup. Is the main factor here 2 seconds sleep, which allows the Javascript to execute completely before parsing the HTML? However, to reiterate, my original run of the code returned complete HTML contents. Could it be that website render just got slower for some reason, since my last attempt at parsing (few months ago)?
@HarleyQuin
The code I ran months ago was same as I posted here, but result was not same. As stated, on my first attempt I got all the HTML contents, while this time I didn't. Also, replacing the parser for lxml parser didn't make a difference. Do you have any idea, from experience, why such difference?
@snippsat
Your code returns all the HTML contents of the page, if I print the soup. Is the main factor here 2 seconds sleep, which allows the Javascript to execute completely before parsing the HTML? However, to reiterate, my original run of the code returned complete HTML contents. Could it be that website render just got slower for some reason, since my last attempt at parsing (few months ago)?