How do I get the HTML of a page after JavaScript has executed?
For testing parser – test page.
I use Nokogiri. But that only gives me the HTML source.
-
1Turn off JavaScript in your browser. Look at the page's source. That's what Nokogiri will see. It takes something with a JavaScript interpreter to do what you want.the Tin Man– the Tin Man2015-04-27 18:27:05 +00:00Commented Apr 27, 2015 at 18:27
Add a comment
|
2 Answers
You need to use some sort of environment that evaluates Javascript, which Nokogiri does not do.
Some examples:
Each of these has pros and cons, without more details I can't really tell you which would be better. You can always just try each one and figure out which one works better for you.
Keep in mind that doing this at scale for sites like Yelp will result in you getting blocked.
Comments
You cannot do it. Javascript is executed by browser. Once rails deliver the html to browser you don't have access to it. I think you want something like Capybara, Selenium, etc, to test your code. See this: http://www.opinionatedprogrammer.com/2011/02/capybara-and-selenium-with-rspec-and-rails-3/
