Python Forum
Best way to process large/complex XML/schema ?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Best way to process large/complex XML/schema ?
#8
Here e demo of using lxml with xPath.
from lxml import etree

root = etree.parse('W2Testfile.xml')
sub = root.xpath('//SubmissionId')[0]
print(sub.text)
for tag in root.xpath('//DependentRelationshipCd'):
    print(tag.text)
00000000000000002222
SON
HALF BROTHER
HALF BROTHER
STEPCHILD
MDRI Wrote:We may have 45K to 50K xml elements to traverse this way.
Not that large would not cause any problems.
MDRI Wrote:Is lxml work as DOM serial parsing ? How will address of pulling all this big XML into DOM?
lxml can operate in either mode,it just depends how your code uses it.
Can either parse the full file into a DOM, or use Sax callbacks to parse serially.
MDRI Wrote:Are there any options in Python do parallel parsing like SAX (Java)?
Yes serval ways eg concurrent.futures

Try some stuff and see how it goes.
Reply


Messages In This Thread
RE: Best way to process large/complex XML/schema ? - by snippsat - May-16-2021, 09:31 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  __init__() got multiple values for argument 'schema' dawid294 4 2,408 Jan-03-2024, 09:42 AM
Last Post: buran
  how to catch schema error? maiya 0 1,872 Jul-16-2021, 08:37 AM
Last Post: maiya
  Missing Schema-Python Question Andwconteh 1 2,522 Jun-16-2021, 01:00 PM
Last Post: Andwconteh
  How to sharing object between multiple process from main process using Pipe Subrata 1 3,679 Sep-03-2019, 09:49 PM
Last Post: woooee
  Avoid output buffering when redirecting large data (40KB) to another process Ramphic 3 3,418 Mar-10-2018, 04:49 AM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020