Python Forum

After several searching and modifies, I still couldn't figure it out. It stops at the 19th in the list.

This is the error. Please help me resolve it.

line 22 in <module> parent_td=[td for td in possible_tds if 'Product' in td.text][0]
Index Error: List index out of range

    from urllib.request import urlopen as uReq
    from urllib.request import Request
    from bs4 import BeautifulSoup as soup
    import re

    ListName = "list"
    OpenList = open(ListName,"r")
    n=int(OpenList.readline())

    for num in range(1,n+1):
         print(num)
         theList=OpenList.readline()

         AccessCME=Request(theList,headers={"User-Agent":"Mozilla/5.0"})
         CMEPage=uReq(AccessCME).read()

         page_soup=soup(CMEPage,"html.parser")
         cme=page_soup.find("div",{"class":"cmeProduct section"})
         FuturesContracts=cme.span.text.strip()

         possible_tds=page_soup.find_all('td',attrs={'class':"prodSpecAtribute"})
         parent_td=[td for td in possible_tds if 'Product' in td.text][0]
         target = parent_td.fetchNextSiblings('td')[0].text.strip()
         first_take=re.sub('CME Globex:\s', '', target)
         BaseSymbol=re.sub(r'CME ClearPort:.*', '', first_take)

         print(FuturesContracts, BaseSymbol)

The link is the list file:

https://drive.google.com/file/d/0BzzXkoI...sp=sharing

Let me know if I need to clarify anything else. Improvements/suggestions are welcome.

Quote: line 22 in <module> parent_td=[td for td in possible_tds if 'Product' in td.text][0]
Index Error: List index out of range

So possible_tds doesn't have any elements, or at least none of them have "Product" in the text.  Try print(possible_tds) and see what's actually there.

Look at the source for link #19. None of the td tags with class prodSpecAttribute have Product in their text. So the list comprehension in line 22 comes back empty, and there is no 0 index, so you get the error.

http://www.cmegroup.com/trading/metals/b...tions.html has no Product in any td.text, so parent_id is empty list and when you try to get element with index 0 you get this error.
There are a lot of possible improvements.

---

EDIT: Sorry for double/triple post :-)