Feb-14-2021, 04:41 PM
With requests.get() the file downloaded is only 3/4 of the file I get by bringing up the page in Firefox and doing a "Save As". It lacks a data item that is visible when reading the page on the screen.
Requests module get() incomplete download
|
Feb-14-2021, 04:41 PM
With requests.get() the file downloaded is only 3/4 of the file I get by bringing up the page in Firefox and doing a "Save As". It lacks a data item that is visible when reading the page on the screen.
Feb-14-2021, 07:09 PM
What exactly do you expect, without showing any code?
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link Create MCV example Debug small programs
Feb-17-2021, 04:51 PM
Your reproach is justified. I understand nothing about Web communications, and do not know where to start. For what it is worth, I took a sample case. I downloaded a page with Requests using this program:
#!/usr/bin/python import requests url='https://www.wsj.com/market-data/quotes/IBM/financials/annual/balance-sheet' r=requests.get(url,allow_redirects=True) open('ibmrqst.html','wb').write(r.content) Then I called up the page with Firefox, right clicked on the page and did "Save Page As..." with "Web Page,complete" and then with "Web Page,HTML only". This obtained the following: Requests download: -rw-r--r-- 1 boba boba 710985 Feb 17 11:19 ibmrqst.html Firefox download complete: -rw-rw-r-- 1 boba boba 527413 Feb 17 11:21 ibmffx.html drwxr-xr-x 2 boba boba 4096 Feb 17 11:21 ibmffx_files Firefox download HTML only: -rw-rw-r-- 1 boba boba 414414 Feb 17 11:23 ibmffxhonly.html I do not know whether any significant values on the page are missing or not; it is just that the sizes of the files are different, even though they are all HTML. I had hoped there was a generally known explanation for such cases. |
|
Possibly Related Threads… | |||||
Thread | Author | Replies | Views | Last Post | |
POST requests - different requests return the same response | Default_001 | 3 | 2,816 |
Mar-10-2022, 11:26 PM Last Post: Default_001 |
|
requests module is not working | varsh | 3 | 5,018 |
Sep-10-2020, 03:53 PM Last Post: buran |
|
ModuleNotFoundError: No module named "requests" | BushFacts | 1 | 6,538 |
Jan-13-2020, 06:49 AM Last Post: perfringo |
|
help about requests download | ggbaby | 1 | 2,874 |
Sep-18-2018, 03:44 AM Last Post: wavic |
|
An Error in Requests Module | pratheep | 3 | 12,875 |
Feb-06-2018, 05:17 PM Last Post: pratheep |
|
Using python requests module and BS4 to login on an Wordpress based website | apollo | 1 | 10,336 |
Feb-06-2018, 01:31 AM Last Post: metulburr |
|
disable proxy with requests module | metulburr | 2 | 11,328 |
Apr-05-2017, 02:54 AM Last Post: metulburr |