1

I have an android java app sending bytes over a socket which is connected to a host machine running a server in Python. I need to receive these bytes as they were sent from the python socket. I see that in Python 'socket.recv' only returns a string. When I send an ASCII string from the java app, I am able to receive the data correctly in the python server, but when I send binary data using java byte, I see the data received is not same. I need to receive raw bytes in Python for my protocol to work correctly. Please point me in right direction.

Code snippet for Sending data on socket:

private void sendFrameMessage(byte[] data) {
        byte[] lengthInfo = new byte[4];
        Log.v(TAG, "sendFrameMessage");

        for(int i=0; i<data.length; i++) {
            Log.v(TAG, String.format("data[%d] = %d", i, data[i]));
        }

        try {
            lengthInfo[0] = (byte) data.length;
            lengthInfo[1] = (byte) (data.length >> 8);
            lengthInfo[2] = (byte) (data.length >> 16);
            lengthInfo[3] = (byte) (data.length >> 24);
            DataOutputStream dos;
            dos = new DataOutputStream(mSocket.getOutputStream());
            dos.write(lengthInfo, 0, 4);
            dos.write(data, 0, data.length);
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

Python Code on receiver side

def recvFrameMessage(self, s):
        recv_count = 4;
        data = s.recv(recv_count)
        if data == 0:
            return None
        total_rx = len(data)
        lenInfo = data
        while total_rx < recv_count:
            data = s.recv(recv_count - total_rx)
            if data == 0:
                return None
            total_rx += len(data)
            lenInfo = lenInfo + data

        recv_count = self.decodeFrameLen(lenInfo)
        logger.info("length = %d" % recv_count)

        data = s.recv(recv_count)
        total_rx = len(data)
        msg = data
        while total_rx < recv_count:
            data = s.recv(recv_count - total_rx)
            if data == 0:
                return None            
            total_rx += len(data)
            msg = msg + data
        logger.info("msg = " + msg)
        for i in range(0, len(msg)-1):
            logger.info("msg[%d] = %s" % (i, msg[i]))
        return msg
Improve this question
1
  • 2
    Could you provide some code of yours? Commented Nov 3, 2013 at 21:10

2 Answers 2

Reset to default
2

@SteveP makes good points for binary data "with some structure", but if this is a plain stream of bytes, in Python 2 simply apply the ord() function to each "character" you get from the socket. For example, if the Java end sends a NUL byte, that will show up on the Python end as the character "\x00", and then:

>>> ord("\x00")
0

To convert a whole string s,

map(ord, s)

returns a list of the corresponding 8-bit unsigned integers.

I'm assuming Python 2 here.

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

Hey, thanks for msp(ord, s), tim. You rock. This was exactly what I needed. Worked like a charm.
Note that in java, the representation for ints is 2's complement.
Isn't it same in python too?
@SteveP, I don't care - bits is bits ;-) Whether a program wants to view them as characters, signed 2's comp, unsigned binary ... is up to the program. @PunitSoni, Python doesn't really have different sizes of integer (Java does). It "acts as if" integers were represented in an unbounded-width 2's-complement representation. It's possible that the Java end will send "byte" -128. That's binary (in Python notation) 0b10000000, and ord() will treat that as +128. If that's a problem, subtract 256 from any ord() result >= 128. Or use struct.unpack() instead of ord().
Yes, in that case, I just found a better solution using python struct.unpack() method. It can interpret the string returned by socket as various C like data types, like signed/unsigned int, long, float etc. That looks easier and scalable solution to this problem
|
1

Reading binary data is perfectly doable, but what if the binary representation from your android app is different than the byte representation on the Python server? From the Python documentation:

It is perfectly possible to send binary data over a socket. The major problem is that not all machines use the same formats for binary data. For example, a Motorola chip will represent a 16 bit integer with the value 1 as the two hex bytes 00 01. Intel and DEC, however, are byte-reversed - that same 1 is 01 00. Socket libraries have calls for converting 16 and 32 bit integers - ntohl, htonl, ntohs, htons where “n” means network and “h” means host, “s” means short and “l” means long. Where network order is host order, these do nothing, but where the machine is byte-reversed, these swap the bytes around appropriately.

Without code and example input/output, this question is going to be really difficult to answer. I assume the issue is that the representation is different. The most likely issue is that Java uses big endian, whereas Python adheres to whatever machine you are running it off of. If your server uses little endian, then you need to account for that. See here for a more thorough explanation on endianness.

Improve this answer

3 Comments

The endianness only comes into play when the data is multi byte, right? I am sending one byte at a time, not 4-byte words. So, I assume, I should receive data in same order as its sent. Correct me if I am wrong. BTW, added code snippets.
@PunitSoni No, consider the representation of the number 10. If you send the bytes from java, you would get the following: 10100000, '00000000', '00000000', '00000000'. But let's just assume that we were only sending the first byte: 10100000. Here, the first 1 is in the least significant bit, but we could easily interpret that bit as being the most significant bit, ie we would get 160, not 10.
@PunitSoni Additionally, for integers, java uses 2's complement, which is another thing to consider.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.