Python Forum
submit element from a list into a post request
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
submit element from a list into a post request
#1
Hello everybody,

I'm a Python beginner and I have no idea on how to solve this problem

I create a variable called list_id with several customer codes inside; then I should use every element of this list separately in a post request (this post request performs a search in a form), scrape the result of the search, make some cleaning and export to a .csv file.

I succeeded in doing it once, but I can't figure out how to create a loop.

This is the code I created so far:
list_id = [ ] #create an empty list where I append all the customer codes I need

for td in soup.find_all('td', {'data-review-id':True}):
    list_id.append(td['data-review-id'].replace('"', ''). replace(r'\/', '/').replace('\\', '').encode("latin-1")) # I extract the customer codes from a web page and append them to my list_id
#print list_id

# I set up the data for my post request:

data2 = {
'reviews_review_search_presenter[ownClientOnlyYN]': 'false',
'reviews_review_search_presenter[userID]': '5228',
'utf8': '\u2713',
'authenticity_token': form3["authenticity_token"], #it takes the authenticity token from elsewhere
'reviews_review_search_presenter[timeOffset]': '0',
'reviews_review_search_presenter[reserveYN]': 'F',
'reviews_review_search_presenter[aktiv]': 'alle' ,
   'reviews_review_search_presenter[meinungstyp]': 'R' ,
   'reviews_review_search_presenter[typus]': '' ,
   'reviews_review_search_presenter[minNote]': '1' ,
   'reviews_review_search_presenter[maxNote]': '5' ,
   'reviews_review_search_presenter[fromDate]': '' ,
   'reviews_review_search_presenter[fromTime]': '' ,
   'reviews_review_search_presenter[toDate]': '' ,
   'reviews_review_search_presenter[toTime]': '' ,
   'reviews_review_search_presenter[fromUpdateDate]': '' ,
   'reviews_review_search_presenter[fromUpdateTime]': '' ,
   'reviews_review_search_presenter[toUpdateDate]': '' ,
   'reviews_review_search_presenter[toUpdateTime]': '' ,
   'reviews_review_search_presenter[artikelNummer]': '' ,
   'reviews_review_search_presenter[masterStyleID]': '' ,
   'reviews_review_search_presenter[digiStyleID]': '' ,
   'reviews_review_search_presenter[id]': list_id[0], #I take only the first customer code in my list_id
   'reviews_review_search_presenter[text]': '' ,
#   'reviews_review_search_presenter[ownClientOnlyYN]': 'false' ,
   'reviews_review_search_presenter[accountNumber]': '' ,
   'reviews_review_search_presenter[name]': '' ,
   'reviews_review_search_presenter[firstName]': '' ,
   'reviews_review_search_presenter[email]': ''
}
response3 = s.post('http://mywebsite', data = data2)
#print(response3.status_code)
page3 = response3.text
#print page
soup2 = BeautifulSoup(page3, 'html.parser')

for table in soup2.find_all(["table", {"class": "customer-info-table"}], limit=1):
    string = table.get_text().replace("\\n", "").replace("\/th", "").replace("\/span", "").replace("\/i", "").replace("\/p", "").replace("\/a", "").replace("\/tr", "").replace("\/div", "").replace("\/td", "").replace("<\/tbody", "").replace("\/table", "").replace("\/textarea", "").replace("\/b","").replace("\/select", "").replace("<\/option", "").replace("<\/label", "").replace("ead", "").replace("<>", "").encode("utf-8")
ext = "Letzte"
fileNameOnly = string[:string.find(ext) + len(ext)]
print fileNameOnly

reviews = []

reviews.append(fileNameOnly)
print reviews
file = open("out.csv", "wb")
file.write(fileNameOnly)
For the moment it works using only the first element from my list_id, but I would like to set up a loop that takes each element from my list_id, make a different post request, append it to the variable "reviews", so that I can download the final .csv file with all the reviews I need together.

Is it possible (and, if yes, how?). Any idea or suggestion would be really appreciated.

Thank you in advance for your help!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
Question Python request (post/get) Drunknmonkie 1 2,627 Jan-19-2023, 02:02 PM
Last Post: prvncpa
  Post request not saving in DRF Dexty 0 893 Jun-03-2022, 12:35 PM
Last Post: Dexty
Brick how to work with 2 or more related tables in a Post request ikurorox 0 1,382 Dec-05-2021, 01:01 AM
Last Post: ikurorox
  Using Python request without selenium on html form with javascript onclick submit but eraosa 0 3,135 Jan-09-2021, 06:08 PM
Last Post: eraosa
  POST request with form data issue web scraping hoff1022 1 2,649 Aug-14-2020, 10:25 AM
Last Post: kashcode
  The correct POST request abhie_lp 5 2,930 Jun-05-2020, 07:27 AM
Last Post: buran
  How to apply post request in python for establishment search for EPFO? rajeev1729 0 1,853 May-01-2020, 04:45 PM
Last Post: rajeev1729
  Handle parameters in POST request for python webserver? JBristow1729 1 7,658 Jul-16-2019, 10:59 PM
Last Post: scidam
  Creating few models with one submit depending on the selected items from multiselect? dmytruk 0 1,855 Mar-18-2019, 03:27 AM
Last Post: dmytruk
  Submit button not working when using form tuannv562 0 5,529 Apr-23-2018, 03:49 AM
Last Post: tuannv562

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020