NodeJS receiving data from PHP server

Question

I'm currently working on a simple NodeJS client that connects to a PHP server using the net classes. In addition, the NodeJS client is working as a Socket.IO server that sends data received from the PHP server to the browsers connected with Socket.IO.

So far, everything is working fine. Yet if I connect with another client to Socket.IO, the PHP server has to send a notification to every connected client. Thus, it sends a JSON-encoded array to the NodeJS client which processes the JSON data (decoding and modifying it a bit).

Now the problem is that sometimes two separate messages sent by the PHP server are concatenated in NodeJS' onData event handling function:

client.on("data", function(data) {

   var msgData = JSON.parse(data.toString("utf8"));
   [...]
}

The variable data now sometimes (not every time!) contains two JSON-strings, such as:

 { "todo":"message", [...] } { "todo":"message", [...] }

This of course results in an exception thrown by the JSON.parse function. I expected two calls of the onData-function with the variable data being:

{ "todo":"message", [...] }

On the PHP server side I have to iterate over an array containing all Socket.IO-connections that are currently served:

foreach($sockets as $id => $client) {

    $nodeJS->sendData($client, array("todo" => "message", [...]);
}

The $nodeJS->sendData-function json-encodes the array and sends it to the NodeJS client:

socket_write($nodeClient, json_encode($dataToSend));

The $nodeJS->sendData function is definitively called two times, as socket_write is.

I now have no idea whether PHP or NodeJS concatenates those two strings. What I want, is that NodeJS calls the onData-handler once for each time the $nodeJS->sendData function is called (e.g. sendData is called twice → the onData-event is fired twice). I could of course add some flag at the end of each json-encoded string and later split them into an array in the onData function. However, I don't like that solution much.

Is there an easier way to accomplish this?

ebohlman · Accepted Answer · 2012-08-23 01:08:07Z

4

It's important to remember that when you're reading from a socket, the data is going to come in arbitrary chunks and its entirely up to your code to split them up into units that are meaningful to process; there is absolutely no guarantee that each chunk will correspond to one meaningful unit.

yannisgu has given you the first part of the solution (terminate each unit with a newline, so your code can tell where it ends): now you need to implement the second part, which is to buffer your incoming data and split it into units.

At initialization, do something like

var buf = '';

and set client's encoding to utf8.

In your "data" handler: [UPDATED: incorporated josh3736's suggestions]

buf += data;
var idx;
while ((idx = buf.indexOf('\n')) >= 0) {
   // there's at least one complete unit buffered up
   var unit = buf.slice(0, idx);
   // extract it
   if (unit.slice(-1) == '\r') {
     // CRLF delimited
     unit = unit.slice(0, -1);
   }
   if (unit.length) {
     // ignore empty strings
     var msgData = JSON.parse(unit);
     [...]
     // process it
   }
   buf = buf.slice(idx +1);
   // discard it
}
// at this point, buf is either empty or contains an incomplete
// unit which will be processed as soon as the rest of it comes in

edited Aug 23, 2012 at 1:08

answered Aug 23, 2012 at 0:00

ebohlman

15k5 gold badges35 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

josh3736 Over a year ago

+1; this is the correct way to handle data coming in a stream. The only things I'd add for completeness' sake would be some extra checks to handle CRLF linebreaks (instead of just LF), and handle empty lines (JSON.parse throws if you give it a zero-length string). I wrote a very similar answer a while back that does those things.

yannisgu · Accepted Answer · 2012-08-22 11:16:16Z

1

Try to add a new line after the JSON-String on the PHP side.

answered Aug 22, 2012 at 11:16

community wiki

yannisgu

1 Comment

lukew Over a year ago

Such as socket_write($nodeClient, json_encode($dataToSend) . "\n")? Tried it with both \n and \r\n - unfortunately this didn't fix the problem.

Collectives™ on Stack Overflow

NodeJS receiving data from PHP server

2 Answers 2

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related