Python Forum
Getting error when I run code more than two times
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Getting error when I run code more than two times
#1
I'm trying to scrape several web sites with a code. I could scrape 2 sites but now code doesn't work. It show me "NameError: name 'inf_rest_name' is not defined" when I try to get information about other website. Can you tell me what can I do to evoid that?... what is happening if it was working perfectly with two sites before?
Thank you so much in advance for your suggestions.
Reply
#2
Without the code, it's really hard to say. It sounds like you have three different web sites, and it's having problems with the third one. My guess would be that inf_rest_name is created in a conditional, such as if, elif, or else. The first two web sites trigger the conditional that creates inf_rest_name, but the third one does not. So you need to give that a default initialization, or you need to reexamine the conditions under which it is initialized.
Craig "Ichabod" O'Brien - xenomind.com
I wish you happiness.
Recommended Tutorials: BBCode, functions, classes, text adventures
Reply
#3
Thank you for your advices.
However I share my code if you have another suggestions.
 with requests.Session() as s:
        for offset in range(40,290,10):
            url = f'https://www.tripadvisor.fr/Restaurant_Review-g187147-d9783452-Reviews-or{offset}-Boutary-Paris_Ile_de_France.html'
            r = s.get(url)
            soup = bs(r.content, 'lxml')
            reviews = soup.select('.reviewSelector')
            ids = [review.get('data-reviewid') for review in reviews]
            r = s.post(
                    'https://www.tripadvisor.fr/OverlayWidgetAjax?Mode=EXPANDED_HOTEL_REVIEWS_RESP&metaReferer=',
                    data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'},
                    headers = {'referer': r.url}
                    )

            soup = bs(r.content, 'lxml')
            if not offset:
                    inf_rest_name = soup.select_one('.heading').text.replace("\n","").strip()
                    rest_eclf = soup.select_one('.header_links a').text.strip()

            for review in soup.select('.reviewSelector'):
                name_client = review.select_one('.info_text > div:first-child').text.strip()
                date_rev_cl = review.select_one('.ratingDate')['title'].strip()
                titre_rev_cl = review.select_one('.noQuotes').text.strip()
                opinion_cl = review.select_one('.partial_entry').text.replace("\n","").strip()
                row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}" , f"{titre_rev_cl}", f"{opinion_cl}"]
                w.writerow(row)
And when I run the code I have
 File "C:/Python/Downloads/webs 3/Sin título1.py", line 37, in <module>
    row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}" , f"{titre_rev_cl}", f"{opinion_cl}"]

NameError: name 'inf_rest_name' is not defined
I don't understant if I used the same code, I just changed the url and the pages number.
Reply
#4
Well, like I said: you are defining inf_rest_name conditionally (on line 16, within the conditional on line 15). Given the range you are looping offset over, that expression should never be False. So why do you have that conditional?
Craig "Ichabod" O'Brien - xenomind.com
I wish you happiness.
Recommended Tutorials: BBCode, functions, classes, text adventures
Reply
#5
so What can I do? ... It's so strange because I scraped two sites webs before, information is well and it didn't show me error. I just changed the url and the number of pages.
Reply
#6
By "changed the number of pages" do you mean you changed line 2? If so, what did you change it to?
Craig "Ichabod" O'Brien - xenomind.com
I wish you happiness.
Recommended Tutorials: BBCode, functions, classes, text adventures
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020