2

The following codes are snippets of a larger software package. I want to understand how the code manages to send objects like numpy arrays, cells. What I wonder about is that the code does not seem to serialize the objects, but it does work. Why does it work?

Python client code (snippet):

class DriverSocket(socket.socket):

   def __init__(self, _socket_interface):
       super(DriverSocket,self).__init__(_sock=_socket_interface)


   def sendpos(self, pos, cell):
      """Sends the position and cell data to the driver.

      Args:
         pos: An array containing the atom positions.
         cell: A cell object giving the system box.

      Raises:
         InvalidStatus: Raised if the status is not Ready.
      """

      if (self.status & Status.Ready):
         try:
            self.sendall(Message("posdata"))
            self.sendall(cell.h)
            self.sendall(cell.ih)
            self.sendall(np.int32(len(pos)/3))
            self.sendall(pos)
         except:
            self.poll()
            return
      else:
         raise InvalidStatus("Status in sendpos was " + self.status)

Receiving code in C (snippet): 

void open_socket_(int *psockfd, int* inet, int* port, char* host)
/* Opens a socket.

Note that fortran passes an extra argument for the string length, but this is
ignored here for C compatibility.

Args:
   psockfd: The id of the socket that will be created.
   inet: An integer that determines whether the socket will be an inet or unix
      domain socket. Gives unix if 0, inet otherwise.
   port: The port number for the socket to be created. Low numbers are often
      reserved for important channels, so use of numbers of 4 or more digits is
      recommended.
   host: The name of the host server.
*/

{
   int sockfd, portno, n;
   struct hostent *server;

   struct sockaddr * psock; int ssock;

   if (*inet>0)
   {  // creates an internet socket
      struct sockaddr_in serv_addr;      psock=(struct sockaddr *)&serv_addr;     ssock=sizeof(serv_addr);
      sockfd = socket(AF_INET, SOCK_STREAM, 0);
      if (sockfd < 0)  error("Error opening socket");

      server = gethostbyname(host);
      if (server == NULL)
      {
         fprintf(stderr, "Error opening socket: no such host %s \n", host);
         exit(-1);
      }

      bzero((char *) &serv_addr, sizeof(serv_addr));
      serv_addr.sin_family = AF_INET;
      bcopy((char *)server->h_addr, (char *)&serv_addr.sin_addr.s_addr, server->h_length);
      serv_addr.sin_port = htons(*port);
      if (connect(sockfd, psock, ssock) < 0) error("Error opening socket: wrong host address, or broken connection");
   }
   else ...



void readbuffer_(int *psockfd, char *data, int* plen)
/* Reads from a socket.

Args:
   psockfd: The id of the socket that will be read from.
   data: The storage array for data read from the socket.
   plen: The length of the data in bytes.
*/

{
   int n, nr;
   int sockfd=*psockfd;
   int len=*plen;

   n = nr = read(sockfd,data,len);

   while (nr>0 && n<len )
   {  nr=read(sockfd,&data[n],len-n); n+=nr; }

   if (n == 0) error("Error reading from socket: server has quit or connection broke");
}

And then there is Fortran code which used the C-socket code

CALL open_socket(socket, inet, port, host)
...
CALL readbuffer(socket, msgbuffer, nat*3*8)

And this receiving code indeed gets a two dimensional arrays and so on. The same works in the opposite direction.

Improve this question

1 Answer 1

Reset to default
1

If you try to send an arbitrary object via send/sendall you will see an exception saying that a bytes-like is expected. So the idea is simple, numpy structures you send provide bytes-like interface, and conversion to bytes leaves just the raw binary data.

The easiest implementation is to inherit from bytes:

class A: pass
sock.sendall(A())  # exception

class B(bytes): pass
sock.sendall(B())  # no exception

However, numpy is a complex framework written in Python, Cython and C. They may use C API to provide the similar functionality.

It's also worth noting that duck-typing doesn't work here as I would expect:

class C:
    def __bytes__(self):
        return bytes()

sock.sendall(C())  # exception
Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

Thanks for your answer. Do you mean you expected that duck-typing would work or not? Also, I programmed a small sample python code where the receiver and sender is a python socket, and sent some numpy arrays. It worked, but the received data was not recovered, I got ▯▯ as value of the received data.
@Jadzia I expected duck-typing to work. As for your sample code, I think it's better to post another question if you're interested. It's possible that not everything in numpy can be send over a network "as is". It's also possible there's something incorrect in your send-receives, but I don't dare to insist on this because I don't see the code.
send/sendall expect a "bytes-like" object which is one that implements the buffer protocol API. Currently the buffer protocol can only be implemented in C. This is why you can send things like array.array and numpy.array and memoryview which do not inherit from bytes, but do implement the C API. There is an open ticket to allow pure python implementations of the buffer protocol, but it's low priority (having been open for nearly 5 years). bugs.python.org/issue13797
@Dunes and Vovanrock: Thanks for your replies, they were very helpful! In the mean time I also managed to send those numpy arrays from python to python and recover the data. This was possible with the method np.fromstring(buffer, np.byte), which returns a proper numpy array again.
@Dunes: You mean send/sendall expect strings (according to the documentation), which are also bytes-like objects, right?
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.