Python Forum
Cant set api response as tree in Element tree
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Cant set api response as tree in Element tree
#1
Hi All,

I can't get the response from the api to be recognised if I try and define tree. Any suggestions would be much appreciated.

import requests
import xml.etree.ElementTree as ET

def getXML():
    url="http://energywatch.natgrid.co.uk/EDP-PublicUI/PublicPI/InstantaneousFlowWebService.asmx"
    
    headers = {'content-type': 'application/soap+xml; charset=utf-8'}
    body ="""<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">
                <soap12:Body>
                <GetInstantaneousFlowData xmlns="http://www.NationalGrid.com/EDP/UI/" />
                </soap12:Body>
                </soap12:Envelope>"""

    response = requests.post(url,data=body,headers=headers)
    return response.content
    
    tree = ET.XML(response) 
    root = tree.getroot()
    root.tag
    print(root.tag)

    for EDPObjectName in root.iter('EDPObjectName'):
        print(EDPObjectName.text)
I can't work out what I'm doing wrong, because If I download the XML file then run the following code, it works:

import xml.etree.ElementTree as ET
tree = ET.parse("C:\\Users\\Hey Arnold\\Documents\\Python Scripts\\new1.xml")

root = tree.getroot()
root.tag
print(root.tag)

for EDPObjectName in root.iter('EDPObjectName'):
    print(EDPObjectName.text)
Reply
#2
All of code is inside the function then after return nothing happens with lines 17-23.
Here a fix an i use BS with lxml as parser,i never use parser in standard library.
import requests
from bs4 import BeautifulSoup

def get_xml():
    url="http://energywatch.natgrid.co.uk/EDP-PublicUI/PublicPI/InstantaneousFlowWebService.asmx"
    headers = {'content-type': 'application/soap+xml; charset=utf-8'}
    body ="""<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">
                <soap12:Body>
                <GetInstantaneousFlowData xmlns="http://www.NationalGrid.com/EDP/UI/" />
                </soap12:Body>
                </soap12:Envelope>"""

    response = requests.post(url,data=body,headers=headers)
    return response.content

response = get_xml()
soup = BeautifulSoup(response, 'lxml')
for item in soup.find_all('EDPObjectName'.lower()):
    print(item.text)
Output:
ALDBROUGH AVONMOUTH BACTON BBL BACTON IC BACTON PERENCO BACTON SEAL BACTON SHELL BARROW SOUTH DYNEVOR ARMS EASINGTON DIMLINGTON EASINGTON LANGELED EASINGTON ROUGH ST .......
Reply
#3
Thanks very much, but is it possible with element tree. It would be great to know what i was doing wrong.
Reply
#4
OK, I think I'm close but no cigar.....yet! Does it look like I have the correct namespace root xpath etc?

import requests
from lxml import etree
    
fromDate = "2018-07-29"

def getXML():
    url="http://energywatch.natgrid.co.uk/EDP-PublicUI/PublicPI/InstantaneousFlowWebService.asmx"
    headers = {'content-type': 'application/soap+xml; charset=utf-8'}
    body ="""<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">
  <soap12:Body>
    <GetInstantaneousFlowData xmlns="http://www.NationalGrid.com/EDP/UI/" />
  </soap12:Body>
</soap12:Envelope>"""

    response = requests.post(url,data=body,headers=headers)
    return response.content

import pandas as pd
df1 = pd.DataFrame(columns=("applicable_at","name","value","created_date"))   
for pd_date in pd.date_range(fromDate, periods=1):
    day = pd_date.strftime('%Y-%m-%d')
     
    root = etree.fromstring(getXML())
    #map prefix 'd' to the default namespace URI
    ns = {'d': 'http://www.nationalgrid.com/EDP/BusinessEntities/Public'}
     
    publication_objects = root.xpath('//d:EDPObjectCollection', namespaces=ns)
     
    for obj in publication_objects:
        name = obj.find('d:EDPObjectName', ns).text
     
        for data in obj.findall('d:EnergyDataList/d:EDPEnergyDataBE', ns):
            applicable_at = pd.to_datetime(data.find('d:ApplicableAt', ns).text)    
            value = float(data.find('d:FlowRate', ns).text)
            created_date = pd.to_datetime(data.find('d:ScheduleTime', ns).text)

        
         
            df1.loc[len(df1) +1] = [applicable_at,name, value,created_date]
Reply
#5
hey_arnold, Have you managed to find a solution?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How and where to store a data for path tree? zayacEBN 1 1,902 Aug-21-2020, 10:14 PM
Last Post: Larz60+
  How to display XML tree structure with Python? sonicblind 13 41,843 Aug-12-2020, 02:05 PM
Last Post: mreshko
  Form add tree comments with mptt m0ntecr1st0 1 2,479 Feb-23-2019, 01:50 PM
Last Post: m0ntecr1st0
  [Tkinter] Does anybody know that how to edit tree view on tkinter Prince_Bhatia 11 14,128 Jul-31-2018, 01:37 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020