Coder Perfect

How to parse XML and count instances of a particular node attribute?


I’m trying to develop a Python script to count instances of a specific node attribute in a database with numerous rows containing XML.

This is how my tree appears:

      <type foobar="1"/>
      <type foobar="2"/>

How can I use Python to retrieve the XML properties “1” and “2”?

Asked by randombits

Solution #1

I recommend ElementTree. Other compatible implementations of the same API exist in the Python standard library, such as lxml and cElementTree; but, in this context, what they mostly add is speed; the ease of programming portion is determined by the API, which ElementTree defines.

Create an Element instance root from the XML, for example, by using the XML function or processing a file with something like:

import xml.etree.ElementTree as ET
root = ET.parse('thefile.xml').getroot()

Or any of the other options listed on ElementTree. Then do anything along these lines:

for type_tag in root.findall('bar/type'):
    value = type_tag.get('foobar')

And comparable coding patterns, which are usually quite easy.

Answered by Alex Martelli

Solution #2

The quickest and most straightforward method is minidom.


        <item name="item1"></item>
        <item name="item2"></item>
        <item name="item3"></item>
        <item name="item4"></item>


from xml.dom import minidom
xmldoc = minidom.parse('items.xml')
itemlist = xmldoc.getElementsByTagName('item')
for s in itemlist:



Answered by Ryan Christensen

Solution #3

BeautifulSoup can be used in the following ways:

from bs4 import BeautifulSoup

      <type foobar="1"/>
      <type foobar="2"/>


[<type foobar="1"></type>, <type foobar="2"></type>]


Answered by YOU

Solution #4

There are numerous options available. If performance and memory utilization are important, cElementTree appears to be a good choice. When compared to merely reading in the file using readlines, it has a very low overhead.

The following table, copied from the cElementTree website, contains the pertinent metrics:

library                         time    space
xml.dom.minidom (Python 2.1)    6.3 s   80000K
gnosis.objectify                2.0 s   22000k
xml.dom.minidom (Python 2.4)    1.4 s   53000k
ElementTree 1.2                 1.6 s   14500k  
ElementTree 1.2.4/1.3           1.1 s   14500k  
cDomlette (C extension)         0.540 s 20500k
PyRXPU (C extension)            0.175 s 10850k
libxml2 (C extension)           0.098 s 16000k
readlines (read as utf-8)       0.093 s 8850k
cElementTree (C extension)  --> 0.047 s 4900K <--
readlines (read as ascii)       0.032 s 5050k   

cElementTree comes packaged with Python, as @jfs pointed out:

Answered by Cyrus

Solution #5

For simplicity, I choose xmltodict.

It converts XML into an OrderedDict.

>>> e = '<foo>
                 <type foobar="1"/>
                 <type foobar="2"/>
        </foo> '

>>> import xmltodict
>>> result = xmltodict.parse(e)
>>> result

OrderedDict([(u'foo', OrderedDict([(u'bar', OrderedDict([(u'type', [OrderedDict([(u'@foobar', u'1')]), OrderedDict([(u'@foobar', u'2')])])]))]))])

>>> result['foo']

OrderedDict([(u'bar', OrderedDict([(u'type', [OrderedDict([(u'@foobar', u'1')]), OrderedDict([(u'@foobar', u'2')])])]))])

>>> result['foo']['bar']

OrderedDict([(u'type', [OrderedDict([(u'@foobar', u'1')]), OrderedDict([(u'@foobar', u'2')])])])

Answered by myildirim

Post is based on