How to open multiple URLs with cURL without delay

Question

I was wondering if it's possible to open multiple URLs with cURL or maybe something else.

I tried this until now.

$urls = array(
          "http://google.com",
          "http://youtube.com",
          );

foreach($urls as $url) {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT,0);
    curl_setopt($ch, CURLOPT_TIMEOUT_MS, 200);
    curl_exec($ch);
    curl_close($ch);
}

The 200ms are there to let the site open fully. Maybe you know any alternatives.

Is it possible to open multiple URLs in PHP at the same time? Not client sided, server side.

I think curl_multi does what you need. Instead of curl_exec you should add the handles to a curl_multi handle and then do a curl_mutli_exec — apokryfos
– apokryfos, Commented Jul 18, 2017 at 16:22

Abraham Tugalov · Accepted Answer · 2017-07-18 16:23:57Z

3

Your solution would be simultaneous cURL HTTP requests.
For faster implementation, you can use this function (thanks to phpied):

function multiRequest($data, $options = array()) {

  // array of curl handles
  $curly = array();
  // data to be returned
  $result = array();

  // multi handle
  $mh = curl_multi_init();

  // loop through $data and create curl handles
  // then add them to the multi-handle
  foreach ($data as $id => $d) {

    $curly[$id] = curl_init();

    $url = (is_array($d) && !empty($d['url'])) ? $d['url'] : $d;
    curl_setopt($curly[$id], CURLOPT_URL,            $url);
    curl_setopt($curly[$id], CURLOPT_HEADER,         0);
    curl_setopt($curly[$id], CURLOPT_RETURNTRANSFER, 1);

    // post?
    if (is_array($d)) {
      if (!empty($d['post'])) {
        curl_setopt($curly[$id], CURLOPT_POST,       1);
        curl_setopt($curly[$id], CURLOPT_POSTFIELDS, $d['post']);
      }
    }

    // extra options?
    if (!empty($options)) {
      curl_setopt_array($curly[$id], $options);
    }

    curl_multi_add_handle($mh, $curly[$id]);
  }

  // execute the handles
  $running = null;
  do {
    curl_multi_exec($mh, $running);
  } while($running > 0);


  // get content and remove handles
  foreach($curly as $id => $c) {
    $result[$id] = curl_multi_getcontent($c);
    curl_multi_remove_handle($mh, $c);
  }

  // all done
  curl_multi_close($mh);

  return $result;
}

And use it like this:

$data = array(
  'http://search.yahooapis.com/VideoSearchService/V1/videoSearch?appid=YahooDemo&query=Pearl+Jam&output=json',
  'http://search.yahooapis.com/ImageSearchService/V1/imageSearch?appid=YahooDemo&query=Pearl+Jam&output=json',
  'http://search.yahooapis.com/AudioSearchService/V1/artistSearch?appid=YahooDemo&artist=Pearl+Jam&output=json'
);
$r = multiRequest($data);

echo '<pre>';
print_r($r);

Hope it helps.
Also read this.

answered Jul 18, 2017 at 16:23

Abraham Tugalov

1,91218 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

BERNARDO Over a year ago

But I still have a little question. It doesn't have to do with the other question above. Is it possible to ignore the reponse time of a websites? Like after five seconds it just goes on and doesn't wait for the response of the website.

Abraham Tugalov Over a year ago

Yep, you still able to use CURLOPT_TIMEOUT_MS. Just apply it after curl_setopt($curly[$id], CURLOPT_RETURNTRANSFER, 1); in that function.

BERNARDO Over a year ago

Thank you very much. Everything is working how I wanted to! I'll give you some credits :) With best regards

Abraham Tugalov Over a year ago

Happy to help ;)

Collectives™ on Stack Overflow

How to open multiple URLs with cURL without delay

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related