Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 submit element from a list into a post request
#1
Hello everybody,

I'm a Python beginner and I have no idea on how to solve this problem

I create a variable called list_id with several customer codes inside; then I should use every element of this list separately in a post request (this post request performs a search in a form), scrape the result of the search, make some cleaning and export to a .csv file.

I succeeded in doing it once, but I can't figure out how to create a loop.

This is the code I created so far:
list_id = [ ] #create an empty list where I append all the customer codes I need

for td in soup.find_all('td', {'data-review-id':True}):
    list_id.append(td['data-review-id'].replace('"', ''). replace(r'\/', '/').replace('\\', '').encode("latin-1")) # I extract the customer codes from a web page and append them to my list_id
#print list_id

# I set up the data for my post request:

data2 = {
'reviews_review_search_presenter[ownClientOnlyYN]': 'false',
'reviews_review_search_presenter[userID]': '5228',
'utf8': '\u2713',
'authenticity_token': form3["authenticity_token"], #it takes the authenticity token from elsewhere
'reviews_review_search_presenter[timeOffset]': '0',
'reviews_review_search_presenter[reserveYN]': 'F',
'reviews_review_search_presenter[aktiv]': 'alle' ,
   'reviews_review_search_presenter[meinungstyp]': 'R' ,
   'reviews_review_search_presenter[typus]': '' ,
   'reviews_review_search_presenter[minNote]': '1' ,
   'reviews_review_search_presenter[maxNote]': '5' ,
   'reviews_review_search_presenter[fromDate]': '' ,
   'reviews_review_search_presenter[fromTime]': '' ,
   'reviews_review_search_presenter[toDate]': '' ,
   'reviews_review_search_presenter[toTime]': '' ,
   'reviews_review_search_presenter[fromUpdateDate]': '' ,
   'reviews_review_search_presenter[fromUpdateTime]': '' ,
   'reviews_review_search_presenter[toUpdateDate]': '' ,
   'reviews_review_search_presenter[toUpdateTime]': '' ,
   'reviews_review_search_presenter[artikelNummer]': '' ,
   'reviews_review_search_presenter[masterStyleID]': '' ,
   'reviews_review_search_presenter[digiStyleID]': '' ,
   'reviews_review_search_presenter[id]': list_id[0], #I take only the first customer code in my list_id
   'reviews_review_search_presenter[text]': '' ,
#   'reviews_review_search_presenter[ownClientOnlyYN]': 'false' ,
   'reviews_review_search_presenter[accountNumber]': '' ,
   'reviews_review_search_presenter[name]': '' ,
   'reviews_review_search_presenter[firstName]': '' ,
   'reviews_review_search_presenter[email]': ''
}
response3 = s.post('http://mywebsite', data = data2)
#print(response3.status_code)
page3 = response3.text
#print page
soup2 = BeautifulSoup(page3, 'html.parser')

for table in soup2.find_all(["table", {"class": "customer-info-table"}], limit=1):
    string = table.get_text().replace("\\n", "").replace("\/th", "").replace("\/span", "").replace("\/i", "").replace("\/p", "").replace("\/a", "").replace("\/tr", "").replace("\/div", "").replace("\/td", "").replace("<\/tbody", "").replace("\/table", "").replace("\/textarea", "").replace("\/b","").replace("\/select", "").replace("<\/option", "").replace("<\/label", "").replace("ead", "").replace("<>", "").encode("utf-8")
ext = "Letzte"
fileNameOnly = string[:string.find(ext) + len(ext)]
print fileNameOnly

reviews = []

reviews.append(fileNameOnly)
print reviews
file = open("out.csv", "wb")
file.write(fileNameOnly)
For the moment it works using only the first element from my list_id, but I would like to set up a loop that takes each element from my list_id, make a different post request, append it to the variable "reviews", so that I can download the final .csv file with all the reviews I need together.

Is it possible (and, if yes, how?). Any idea or suggestion would be really appreciated.

Thank you in advance for your help!
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Receiving Werkzeug. exception. Bad Request error when using GET with an endpoint robogeek 0 119 Jul-02-2019, 01:04 PM
Last Post: robogeek
  How to send unicode string encoded in utf-8 in http request in Python MaverinCode 0 350 Apr-25-2019, 05:25 PM
Last Post: MaverinCode
  Creating few models with one submit depending on the selected items from multiselect? dmytruk 0 193 Mar-18-2019, 03:27 AM
Last Post: dmytruk
  Is it possible to perform a PUT request by passing a req body instead of an ID ary 0 214 Feb-20-2019, 05:55 AM
Last Post: ary
  Failed to locate element gahhon 4 377 Feb-18-2019, 02:23 PM
Last Post: gahhon
  Python + request from specific website - please help hoff1022 8 526 Feb-14-2019, 06:52 PM
Last Post: buran
  Flask rest api How to retrieve json request raysefo 4 726 Jan-20-2019, 06:46 PM
Last Post: raysefo
  Python Selenium getting table element trengan 2 525 Dec-31-2018, 03:02 PM
Last Post: trengan
  Click Element if displayed using Selenium and Python giaco__mar 1 443 Dec-27-2018, 06:19 PM
Last Post: metulburr
  Flask redirect(request.url) returns None iFunKtion 1 1,904 Nov-27-2018, 10:50 PM
Last Post: nilamo

Forum Jump:


Users browsing this thread: 1 Guest(s)