Oct-20-2018, 05:09 PM
I've hit a bit of a brick wall, I'm sure there's a way around it, but my head is swimming in python syntax at the moment and I just can't see it!
My app is in two parts, a socket server that perversely sits on my clients, and a socket client that sits on my server.
When it's working ok the client logs onto all the servers, sends a code, the servers generate a data report and send it back. Lovely Jubly.
Except when a socket can't be opened. When the socket.connect(blah) fails for some reason the app stalls until the time out and things gradually start to fall apart.
So, I put all the data requests (socket connects) in new threads, so if one sits there the others don't get hung. This then needs some kind of marshalling as it's possible to go around the list and try to start a second thread on the same problem node/client.
So I wrote a class and some add/remove code to track open threads and make sure this didn't happen and now the app has such a large overhead I might as well me sending the data to a database, which is what I was trying to avoid in the first place!
I know I've asked a similar question before, and I have to admit I haven't checked responses yet, I thought I'd ramble while this was clear in my head.
If anyone out there has done anything similar, could you please let me know what you did in the end?
Outline:
Server/header raspberry pi
Clients raspberry pi
Number of clients/nodes 128
Data collection every 1 or 2 seconds. Could be increased with the server is working hard.
Data to be collected: core temperature, cpu load, memory stats, swap file stats, disk stats and some other bit of data which could be a one off call.
I'm going to write the database approach tonight, I'm very used to database work so this will be really easy, then try to gauge which will be the most heavy. Even with the database approach there will have to be some live communication to change timing, data required etc.
I don't need anything writing, I'm enjoying the learning, I just want to end up with the most sensible solution in place. While it's not mission critical I'd sooner do it right.
Many thanks.
My app is in two parts, a socket server that perversely sits on my clients, and a socket client that sits on my server.
When it's working ok the client logs onto all the servers, sends a code, the servers generate a data report and send it back. Lovely Jubly.
Except when a socket can't be opened. When the socket.connect(blah) fails for some reason the app stalls until the time out and things gradually start to fall apart.
So, I put all the data requests (socket connects) in new threads, so if one sits there the others don't get hung. This then needs some kind of marshalling as it's possible to go around the list and try to start a second thread on the same problem node/client.
So I wrote a class and some add/remove code to track open threads and make sure this didn't happen and now the app has such a large overhead I might as well me sending the data to a database, which is what I was trying to avoid in the first place!
I know I've asked a similar question before, and I have to admit I haven't checked responses yet, I thought I'd ramble while this was clear in my head.
If anyone out there has done anything similar, could you please let me know what you did in the end?
Outline:
Server/header raspberry pi
Clients raspberry pi
Number of clients/nodes 128
Data collection every 1 or 2 seconds. Could be increased with the server is working hard.
Data to be collected: core temperature, cpu load, memory stats, swap file stats, disk stats and some other bit of data which could be a one off call.
I'm going to write the database approach tonight, I'm very used to database work so this will be really easy, then try to gauge which will be the most heavy. Even with the database approach there will have to be some live communication to change timing, data required etc.
I don't need anything writing, I'm enjoying the learning, I just want to end up with the most sensible solution in place. While it's not mission critical I'd sooner do it right.
Many thanks.