How to parse html source code with ruby/nokogiri?

Question

I've successfully used ruby (1.8) and nokogiri's css parsing to pull out front facing data from web pages.

However I now need to pull out some data from a series of pages where the data is in the "meta" tags in the source code of the page.

One of the lines I need is the following:

<meta name="geo.position" content="35.667459;139.706256" />

I've tried using xpath put haven't been able to get it right.

Any help as to what syntax is needed would be much appreciated.

Thanks

You say "I've tried using xpath put haven't been able to get it right." Show us what you have tried so that we can help you do it right. — Andy Lester
– Andy Lester, Commented Oct 27, 2010 at 4:58
Thx Andy - just various ways of saying '//meta[blah]' etc. I just couldn't get the syntax correct to pull it out. I really wanted to do it with the css selector and now I know how. — rollbahn
– rollbahn, Commented Oct 27, 2010 at 5:32

Jordan Running · Accepted Answer · 2010-10-27 05:08:15Z

2

This is a good case for a CSS attribute selector. For example:

doc.css('meta[name="geo.position"]').each do |meta_tag|
  puts meta_tag['content'] # => 35.667459;139.706256
end

The equivalent XPath expression is almost identical:

doc.xpath('//meta[@name = "geo.position"]').each do |meta_tag|
  puts meta_tag['content'] # => 35.667459;139.706256
end

edited Oct 27, 2010 at 5:08

answered Oct 27, 2010 at 5:02

Jordan Running

107k18 gold badges193 silver badges189 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

rollbahn Over a year ago

Wow thanks I had no idea you could use the css selector for meta tags too. If I wanted to get the lat/long from the js does the same apply? <script type="text/javascript"> //<![CDATA[ function onLoad() { if (GBrowserIsCompatible()) { var map = new GMap2(document.getElementById("map")); map.addControl(new GSmallMapControl()); var point1 = new GLatLng(35.667459, 139.706256); map.setCenter(point1, 15, G_NORMAL_MAP); var marker = new GMarker(point1,{clickable:false}); map.addOverlay(marker); } } //]]> </script>

the Tin Man Over a year ago

No, Nokogiri doesn't do Javascript. You could extract the Javascript from the HTML using Nokogiri, then use a regex to get the lat/long. ` doc.at('script').content[/GLatLng\(([^)]+)\)/,1] # => "35.667459, 139.706256"` for instance.

rollbahn Over a year ago

Aha okay awesome thanks very much for your help - that has really made things much clearer.

the Tin Man · Accepted Answer · 2010-10-27 05:13:37Z

1

require 'nokogiri'

doc = Nokogiri::HTML('<meta name="geo.position" content="35.667459;139.706256" />')
doc.at('//meta[@name="geo.position"]')['content'] # => "35.667459;139.706256"

answered Oct 27, 2010 at 5:13

the Tin Man

161k44 gold badges222 silver badges308 bronze badges

Collectives™ on Stack Overflow

How to parse html source code with ruby/nokogiri?

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related