Downloading CSV from a website

[bmiller12]

Hello all,

I ran into a few issues when dealing with Python and CSV files.
1. How do I automatically login to a website?
2. How do I automatically export a CSV file from the website
- The CSV file downloads onto my computer
- I need an updated CSV file whenever I alter my data
3. And how do I put the CSV file onto Python
I've been trying to log onto GarminConnect, to upload my activities in the CSV file, and keep running into issues either in the code itself, or the websites HTML code.

Please help!

[Larz60+]

Please show the following

• Your code. (X out any passwords, etc.)
  
• Error messages (complete and unaltered)
  
• Anything else that might prove helpful
  

Thank you.

[bmiller12]

Of course, This is what I have currently when trying to get the HTML to then update it to tell me the stats from the website instead of from the downloadable CSV, this code printed the HTML:

import requests
import csv
import pandas as pd 

login_url = 'https://connect.garmin.com/signin/'
request_url = 'https://connect.garmin.com/modern/activities#'

garmin = { 
  'username':'[email protected]',
  'password':'xxxxxxx'
}

with requests.session() as session:
  post = session.post(login_url, data = garmin)
  r = session.get(request_url)
  decoded_content = r.content.decode('utf-8')

Here's another code I tried:

import numpy as np
import pandas as pd
import mechanicalsoup as ms

browser = ms.StatefulBrowser()
browser.open('https://connect.garmin.com/signin/')
print(browser.get_url())
browser.select_form('form method["post"]')
browser['username'] = '[email protected]'
browser['password'] = 'xxxxx'
browser.submit_selected()
print(browser.get_url()) 

that code gave this error:

Error:
File "main.py", line 8, in <module>
    browser.select_form('form method["post"]')
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/mechanicalsoup/stateful_browser.py", line 201, in select_form
    found_forms = self.get_current_page().select(selector,
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/bs4/element.py", line 1869, in select
    results = soupsieve.select(selector, self, namespaces, limit, **kwargs)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/soupsieve/__init__.py", line 98, in select
    return compile(select, namespaces, flags, **kwargs).select(tag, limit)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/soupsieve/__init__.py", line 62, in compile
    return cp._cached_css_compile(pattern, namespaces, custom, flags)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/soupsieve/css_parser.py", line 208, in _cached_css_compile
    CSSParser(pattern, custom=custom_selectors, flags=flags).process_selectors(),
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/soupsieve/css_parser.py", line 1043, in process_selectors
    return self.parse_selectors(self.selector_iter(self.pattern), index, flags)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/soupsieve/css_parser.py", line 894, in parse_selectors
    key, m = next(iselector)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/soupsieve/css_parser.py", line 1036, in selector_iter
    raise SelectorSyntaxError(msg, self.pattern, index)
soupsieve.util.SelectorSyntaxError: Malformed attribute selector at position 11
  line 1:
form method["post"]

Thank you for looking

Larz60+ write Nov-27-2020, 04:55 PM:
Please post all code, output and errors (it it's entirety) between their respective tags. Refer to BBCode help topic on how to post. Use the "Preview Post" button to make sure the code is presented as you expect before hitting the "Post Reply/Thread" button.

I fixed for you this time. Please use bbcode tags on future posts.

[sridhar]

Hello all,

I ran into a few issues when dealing with Python and CSV files.
1. How do I automatically login to a website?
2. How do I automatically export a CSV file from the website
- The CSV file downloads onto my computer
- I need an updated CSV file whenever I alter my data
3. And how do I put the CSV file onto Python
I've been trying to log onto GarminConnect, to upload my activities in the CSV file, and keep running into issues either in the code itself, or the websites HTML code.

Please help!

On looking your requirements that you have mentioned above ,You can automate this process using Selenium Package in Python.
You need to find the X-path/id/class of respective Export option (that's not a big deal) you can automates this process.

I would suggest you to Go through Python Selenium Documentation and get some idea about this package ,later it'll be easy to do this.

[Larz60+]

There is a tutorial on this forum that will get you started quickly with selenium (and other web scraping methods) only takes a few hours.
web scraping part 1
web scraping part 2