Python sockets in a multithreaded program with classes/objects interfere with each other

Question

As an aid to my work, I am developing a communication simulator between devices, based on TCP sockets, in Python 3.12 (with an object oriented approach). Basically, the difference between a SERVER type communication channel rather than a CLIENT type is merely based on the way by which the sockets are instantiated: respectively, the server ones listen/accept for connection requests while the client ones actively connect to their endpoint. Once the connection is established, either party can begin to transmit something, which the other receives and processes and then responds (this on the same socket pair of course). As you can see. this simulator has a simple interface based on Tkinter You can create up to 4 channels n a grid layout, in this case we have two:

When the user clicks on CONNECT button, this is what happens in the listener of that button in the frame class:

class ChannelFrame(tk.Frame):

        channel = None #istance of channel/socket type
        
        def connectChannel(self):
            port = self.textPort.get();
            if self.socketType.get() == 'SOCKET_SERVER':
                self.channel = ChannelServerManager(self,self.title,port)
            elif self.socketType.get() == 'SOCKET_CLIENT':
                ipAddress = self.textIP.get()
                self.channel = ChannelClientManager(self,self.title,ipAddress,port)

Then I have an implementation of a channel of type Server and one for type Client. Their constructors basically collect the received data and create a main thread whose aim is to create socket and then:

1a) connect to the counterpart in case of socket client

1b) waiting for requests of connections in case of socket server

2.) enter a main loop using select.select and trace in the text area of their frame the received and sent data

Here is the code for main thread Client

class ChannelClientManager():

        establishedConn = None
        receivedData = None
        eventMainThread = None #set this event when user clicks on DISCONNECT button
        
        def threadClient(self):
            self.socketsInOut.clear()
            self.connected = False
            
            while True:
                if (self.eventMainThread.is_set()):
                    print(f"threadClient() --> ChannelClient {self.channelId}: Socket client requested to shut down, exit main loop")
                    break;
                
                
                if(not self.connected):
                    try :
                        self.establishedConn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                        self.establishedConn.connect((self.ipAddress, int(self.port)))
                        self.channelFrame.setConnectionStateChannel(True)
                        self.socketsInOut.append(self.establishedConn)
                        self.connected = True
                    #keep on trying to connect to my counterpart until I make it
                    except socket.error as err:
                        print(f'socket.error threadClient() --> ChannelClient {self.channelId}: Error while connecting to server: {err}')
                        time.sleep(0.5)
                        continue
                    except socket.timeout as sockTimeout:
                        print(f'socket.timeout threadClient() -->  ChannelClient {self.channelId}: Timeout while connecting to server: {sockTimeout}')
                        continue
                    except Exception as e:
                        print(f'Exception on connecting threadClient() -->  ChannelClient {self.channelId}: {e}')
                        continue
                    
                
                if(self.connected):
                    try:
                        
                        r, _, _ = select.select(self.socketsInOut, [], [], ChannelClientManager.TIMEOUT_SELECT)
                        
                        if len(r) > 0: #socket ready to be read with incoming data
                            for fd in r:
                                data = fd.recv(1)
                                if data:
                                    self.manageReceivedDataChunk(data)
                                else:
                                    print(f"ChannelClient {self.channelId}: Received not data on read socket, server connection closed")
                                    self.closeConnection()
                        else:
                            #timeout
                            self.manageReceivedPartialData()
                    except ConnectionResetError as crp:
                        print(f"ConnectionResetError threadClient() --> ChannelClient {self.channelId}: {crp}")
                        self.closeConnection()
                    except Exception as e:
                        print(f'Exception on selecting threadClient() -->  ChannelClient {self.channelId}: {e}')

Here is the code for main thread Server

class ChannelServerManager():
    
    socketServer = None #user to listen/accept connections
    establishedConn = None #represents accepted connections with the counterpart
    receivedData = None
    eventMainThread = None
    socketsInOut = []

    def __init__(self, channelFrame, channelId, port):
        self.eventMainThread = Event()
        self.socketsInOut.clear()
        self.socketServer = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.socketServer.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.socketServer.bind(('', int(port))) #in ascolto qualsiasi interfaccia di rete, se metto 127.0.0.1 starebbe in ascolto solo sulla loopback
        self.socketServer.listen(1) #accepting one connection from client
        self.socketsInOut.append(self.socketServer)
        
        self.mainThread = Thread(target = self.threadServer)
        self.mainThread.start()


    def threadServer(self): 
            self.receivedData = ''
            
            while True:
                if (self.eventMainThread.is_set()):
                    print("threadServer() --> ChannelServer is requested to shut down, exit main loop\n")
                    break;
                
                try:
                    r, _, _ = select.select(self.socketsInOut, [], [], ChannelServerManager.TIMEOUT_SELECT)
                    
                    if len(r) > 0: #socket pronte per essere lette
                        for fd in r:
                            if fd is self.socketServer:
                                
                                #if the socket ready is my socket server, then we have a client wanting to connect --> let's accept it
                                clientsock, clientaddr = self.socketServer.accept()
                                self.establishedConn = clientsock
                                print(f"ChannelServer {self.channelId} is connected from client address {clientaddr}")
                                self.socketsInOut.append(clientsock)
                                self.channelFrame.setConnectionStateChannel(True)
                                self.receivedData = ''
                            elif fd is self.establishedConn:
                                data = fd.recv(1)
                                if not data:
                                    print(f"ChannelServer {self.channelId}: Received not data on read socket, client connection closed")
                                    self.socketsInOut.remove(fd)
                                    self.closeConnection()
                                else:
                                    self.manageReceivedDataChunk(data)
                    else: #timeout
                        self.manageReceivedPartialData()
                except Exception as e:
                    print(f"Exception threadServer() --> ChannelServer {self.channelId}: {traceback.format_exc()}")

I don't know why, but this frames/sockets appear to interfere with each other or "share data". Or, disconnecting and closing a channel from its button in its own frame also causes the other one into error, or the other one closes/crashes too. These two frames/objects should each live their own life and move forward with their counterpart as long as it is connected, instead they interfere. As you can see from this screenshot:

By a medical device (which is server), I am sending this data

<VT>MSH|^~\&|KaliSil|KaliSil|AM|HALIA|20240130182136||OML^O33^OML_O33|1599920240130182136|P|2.5<CR>PID|1||A20230522001^^^^PI~090000^^^^CF||ESSAI^Halia||19890522|M|||^^^^^^H|||||||||||||||<CR>PV1||I||||||||||||A|||||||||||||||||||||||||||||||<CR>SPM|1|072401301016^072401301016||h_san^|||||||||||||20240130181800|20240130181835<CR>ORC|NW|072401301016||A20240130016|saisie||||20240130181800|||^^|CP1A^^^^^^^^CP1A||20240130182136||||||A^^^^^ZONA<CR>TQ1|1||||||||0||<CR>OBR|1|072401301016||h_GLU_A^^T<CR>OBX|1|NM|h_GLU_A^^T||||||||||||||||<CR>BLG|D<CR><FS>

only to channel on port 10001 but part of this data is received on one socket client, other part on the other (right) socket client. This is not a problem of rendering the text in the right frame, also the log of the received data shows that some data is received in Channel 0 and some other data in Channel 1. Why does this happen? Instead, I start 2 instances of the simulator with only one channel each, then everything works perfectly but this defeats our purpose of being able to work up to 4 channels in parallel from a single window. Do you have any ideas? The first time I had implemented ChannelServerManager and ChannelClientManager as extended from an ChannelAbstractManager with common methods and data structures, based on Python library ABC Then I read that inheritance in Python is not the same as in Java, so I thought the different instances were sharing some attributes. I removed the abstract class and replicated the code and resources in both classes but this has not solved. Any suggestions?

BoppreH · Accepted Answer · 2024-02-02 18:52:48Z

4

Then I read that inheritance in Python is not the same as in Java

Thanks, that was a good tip to find the issue! While not an inheritance issue, I think you're seeing problems caused by this Java-ism:

class ChannelClientManager():

    establishedConn = None
    receivedData = None
    eventMainThread = None #set this event when user clicks on DISCONNECT button
        
...

class ChannelServerManager():
    
    socketServer = None #user to listen/accept connections
    establishedConn = None #represents accepted connections with the counterpart
    receivedData = None
    eventMainThread = None
    socketsInOut = []

    def __init__(self, channelFrame, channelId, port):
        ...

In Python you don't need to declare your attributes in advance, you should just assign to them directly in your __init__ method (constructor). All these variables you're declaring are actually class attributes and hence shared between all instances like you suspected.

It might not be obvious because when you do self.establishedConn = ... you're also creating an instance attribute that overrides the visibility of the class attribute, so you never actually access the shared values.

With one exception:

    socketsInOut = []

    def __init__(self, channelFrame, channelId, port):
        ...
        self.socketsInOut.clear()
        ...
        self.socketsInOut.append(self.socketServer)

Because you never assigned self.socketsInOut, all instances are instead accessing the (shared) class attribute. After a small startup period, all channels end up sending and receiving messages on the same socket list, hence the messages being split.

The fix is to remove all those unnecessary attribute "declarations", and add a missing one for socketsInOut:

class ChannelClientManager():

    def threadClient(self):
        self.socketsInOut = [] # Change here.
        ...

class ChannelServerManager():

    def __init__(self, channelFrame, channelId, port):
        self.eventMainThread = Event()
        self.socketsInOut = [] # And here.
        self.socketServer = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        ...

And to save you time debugging another well known Python wart, default arguments are also shared between calls. So never declare a method with a mutable default argument, like def foo(items=[]): ....

edited Feb 2, 2024 at 18:52

answered Feb 2, 2024 at 18:46

BoppreH

10.3k5 gold badges40 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

SagittariusA Over a year ago

Thank you dear, at last it works. So, this time too I can tell my boss that StackOverflow saved my life. After almost 8 years of working in Java, I actually took it for granted that Python classes worked the same. Many thanks for your kind explanation. Should I want to re-order my code and re-introduce the abstract class via the ABC library of Python, in order to avoid sharing of attributes, do I have to use those particular annotations like @property, @abstractmethod, @establishedConn.setter? stackoverflow.com/questions/13646245/…

BoppreH Over a year ago

@SagittariusA Python OOP is powerful, but it's uncommon to use all available features like in Java. My suggestion is to use @property only for values calculated on demand, and I've only used @attribute.setter a handful of times ever. Abstract classes are ok, but only if you expect somebody to mistakenly instantiate them. Otherwise, putting "Base" on the class name and raising NotImplementedError on abstract methods is usually enough. Regarding attributes, you don't need base classes for that, leave as-is.

SagittariusA Over a year ago

thank you, so this is ok for common attributes. But what about common methods which are the most part of the code? Because, once the connection as been established, then both the sending and receiving and tracing of messages consist all in the same code. Should I leave them replicated in all classes?

BoppreH Over a year ago

@SagittariusA No, in that case a base class is fine. But you don't necessarily need the @abstractmethod annotation, it's only used to prevent some mistakes. Python is all about runtime checks anyway. I personally prefer raising NotImplementedError in "abstract methods".

SagittariusA Over a year ago

Really thank you! I will try my best to refactor the code according to your suggestions :)

Collectives™ on Stack Overflow

Python sockets in a multithreaded program with classes/objects interfere with each other

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related