How to parse an xml feed using python?

Question

I am trying to parse this xml (http://www.reddit.com/r/videos/top/.rss) and am having troubles doing so. I am trying to save the youtube links in each of the items, but am having trouble because of the "channel" child node. How do I get to this level so I can then iterate through the items?

#reddit parse
reddit_file = urllib2.urlopen('http://www.reddit.com/r/videos/top/.rss')
#convert to string:
reddit_data = reddit_file.read()
#close file because we dont need it anymore:
reddit_file.close()

#entire feed
reddit_root = etree.fromstring(reddit_data)
channel = reddit_root.findall('{http://purl.org/dc/elements/1.1/}channel')
print channel

reddit_feed=[]
for entry in channel:   
    #get description, url, and thumbnail
    desc = #not sure how to get this

    reddit_feed.append([desc])

Community · Accepted Answer · 2013-04-25 06:49:09Z

8

You can try findall('channel/item')

import urllib2
from xml.etree import ElementTree as etree
#reddit parse
reddit_file = urllib2.urlopen('http://www.reddit.com/r/videos/top/.rss')
#convert to string:
reddit_data = reddit_file.read()
print reddit_data
#close file because we dont need it anymore:
reddit_file.close()

#entire feed
reddit_root = etree.fromstring(reddit_data)
item = reddit_root.findall('channel/item')
print item

reddit_feed=[]
for entry in item:   
    #get description, url, and thumbnail
    desc = entry.findtext('description')  
    reddit_feed.append([desc])

edited Apr 25, 2013 at 6:49

CommunityBot

11 silver badge

answered Oct 14, 2012 at 3:41

Himanshu

2,4743 gold badges26 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Gilles Quénot · Accepted Answer · 2012-10-14 05:26:06Z

4

I wrote that for you using Xpath expressions (tested successfully ):

from lxml import etree
import urllib2

headers = { 'User-Agent' : 'Mozilla/5.0' }
req = urllib2.Request('http://www.reddit.com/r/videos/top/.rss', None, headers)
reddit_file = urllib2.urlopen(req).read()

reddit = etree.fromstring(reddit_file)

for item in reddit.xpath('/rss/channel/item'):
    print "title =", item.xpath("./title/text()")[0]
    print "description =", item.xpath("./description/text()")[0]
    print "thumbnail =", item.xpath("./*[local-name()='thumbnail']/@url")[0]
    print "link =", item.xpath("./link/text()")[0]
    print "-" * 100

edited Oct 14, 2012 at 5:26

answered Oct 14, 2012 at 3:41

Gilles Quénot

188k43 gold badges232 silver badges229 bronze badges

Collectives™ on Stack Overflow

How to parse an xml feed using python?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related