Python Forum

Full Version: submit element from a list into a post request
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hello everybody,

I'm a Python beginner and I have no idea on how to solve this problem

I create a variable called list_id with several customer codes inside; then I should use every element of this list separately in a post request (this post request performs a search in a form), scrape the result of the search, make some cleaning and export to a .csv file.

I succeeded in doing it once, but I can't figure out how to create a loop.

This is the code I created so far:
list_id = [ ] #create an empty list where I append all the customer codes I need

for td in soup.find_all('td', {'data-review-id':True}):
    list_id.append(td['data-review-id'].replace('"', ''). replace(r'\/', '/').replace('\\', '').encode("latin-1")) # I extract the customer codes from a web page and append them to my list_id
#print list_id

# I set up the data for my post request:

data2 = {
'reviews_review_search_presenter[ownClientOnlyYN]': 'false',
'reviews_review_search_presenter[userID]': '5228',
'utf8': '\u2713',
'authenticity_token': form3["authenticity_token"], #it takes the authenticity token from elsewhere
'reviews_review_search_presenter[timeOffset]': '0',
'reviews_review_search_presenter[reserveYN]': 'F',
'reviews_review_search_presenter[aktiv]': 'alle' ,
   'reviews_review_search_presenter[meinungstyp]': 'R' ,
   'reviews_review_search_presenter[typus]': '' ,
   'reviews_review_search_presenter[minNote]': '1' ,
   'reviews_review_search_presenter[maxNote]': '5' ,
   'reviews_review_search_presenter[fromDate]': '' ,
   'reviews_review_search_presenter[fromTime]': '' ,
   'reviews_review_search_presenter[toDate]': '' ,
   'reviews_review_search_presenter[toTime]': '' ,
   'reviews_review_search_presenter[fromUpdateDate]': '' ,
   'reviews_review_search_presenter[fromUpdateTime]': '' ,
   'reviews_review_search_presenter[toUpdateDate]': '' ,
   'reviews_review_search_presenter[toUpdateTime]': '' ,
   'reviews_review_search_presenter[artikelNummer]': '' ,
   'reviews_review_search_presenter[masterStyleID]': '' ,
   'reviews_review_search_presenter[digiStyleID]': '' ,
   'reviews_review_search_presenter[id]': list_id[0], #I take only the first customer code in my list_id
   'reviews_review_search_presenter[text]': '' ,
#   'reviews_review_search_presenter[ownClientOnlyYN]': 'false' ,
   'reviews_review_search_presenter[accountNumber]': '' ,
   'reviews_review_search_presenter[name]': '' ,
   'reviews_review_search_presenter[firstName]': '' ,
   'reviews_review_search_presenter[email]': ''
response3 ='http://mywebsite', data = data2)
page3 = response3.text
#print page
soup2 = BeautifulSoup(page3, 'html.parser')

for table in soup2.find_all(["table", {"class": "customer-info-table"}], limit=1):
    string = table.get_text().replace("\\n", "").replace("\/th", "").replace("\/span", "").replace("\/i", "").replace("\/p", "").replace("\/a", "").replace("\/tr", "").replace("\/div", "").replace("\/td", "").replace("<\/tbody", "").replace("\/table", "").replace("\/textarea", "").replace("\/b","").replace("\/select", "").replace("<\/option", "").replace("<\/label", "").replace("ead", "").replace("<>", "").encode("utf-8")
ext = "Letzte"
fileNameOnly = string[:string.find(ext) + len(ext)]
print fileNameOnly

reviews = []

print reviews
file = open("out.csv", "wb")
For the moment it works using only the first element from my list_id, but I would like to set up a loop that takes each element from my list_id, make a different post request, append it to the variable "reviews", so that I can download the final .csv file with all the reviews I need together.

Is it possible (and, if yes, how?). Any idea or suggestion would be really appreciated.

Thank you in advance for your help!