How to get webcontent that is loaded by JavaScript using cURL?

Question

I am trying to grab the score table from a website. The problem is that the site is using javascript that apparently loads on document.ready.

For that reason I don't seem to succeed getting the table and all I get is the buttons at the top of that page.

Any suggestions or ideas on how to solve this?

Are you tried file_get_contents ??

Hackerman
– Hackerman

2013-12-12 21:00:36 +00:00
Commented Dec 12, 2013 at 21:00 — Hackerman
– Hackerman, Commented Dec 12, 2013 at 21:00
Try this: stackoverflow.com/a/28506533/3063226

Heitor
– Heitor

2017-02-10 19:49:08 +00:00
Commented Feb 10, 2017 at 19:49 — Heitor
– Heitor, Commented Feb 10, 2017 at 19:49

Vivin Paliath · Accepted Answer · 2013-12-12 21:00:52Z

30

cURL will only get you the markup of the page. It won't load any additional resources or process the page. You probably want to look at something like PhantomJS for this. PhantomJS is a headless WebKit browser. It has its own API that lets you "script" behavior. So you can tell PhantomJS to load the page and dump out the data you need.

answered Dec 12, 2013 at 21:00

Vivin Paliath

95.8k42 gold badges230 silver badges302 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

m33ts4k0z Over a year ago

Thanks for your answer. I will need to run this as JavaScript and then save the dump to a PHP variable using the exec command. Is that correct?

Vivin Paliath Over a year ago

You don't have to, actually. You can run it directly from the command line. But if you are using this to display it on a website, then yes you can use exec from PHP.

Lasse · Accepted Answer · 2013-12-12 21:03:28Z

6

Depending on what exactly you're trying to do, you could just fetch the JSON data that makes the table:

http://www.opap.gr/web/services/rs/betting/availableBetGames/sport/program/4100/0/sport-1.json?localeId=el_GR

answered Dec 12, 2013 at 21:03

Lasse

1,6961 gold badge15 silver badges27 bronze badges

3 Comments

m33ts4k0z Over a year ago

Thanks for your answer. I will mark Vivins answer as the answer to my question but yours was also really good. I wonder how did you find the json call?

Lasse Over a year ago

@m33ts4k0z: No problem. I found it by using the "Network" tab of Chrome's Developer Tools (The same is possible in other browsers as well). Since document.ready was involved I guessed that the page probably loaded the information externally.

niCk cAMel Over a year ago

@PradeepKumarPrabaharan 3 years later LOL

Collectives™ on Stack Overflow

How to get webcontent that is loaded by JavaScript using cURL?

2 Answers 2

2 Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related