Python Forum

Hello,

i need help with implementation Queue and maybe Lock in multithreading operation. Code below will print results i want, but i want to return it as object for further operations. if there is option to solve this with multiprocessing please how to apply it to this concrete code.

self.query_sort() ## Returns Dictionary with ip addresses scope {"9" : "192.168.9.0", "192.168.9.1", ....} it's about 400 IP's in 7 different scopes

self.list_result = []

def get_network_data(self, ixx):
        for ix in ixx:
                sshh = SshConnection(ix, self.username, self.password) # Connecting to SSH ix is ip address
            try:
                out1, err1 = sshh.ssh_command("ioreg -c IOPlatformExpertDevice "
                                              "-d 2 | awk '/IOPlatformSerialNumber/'")  # Get computer  serial number
                serial_number = out1[0].strip().split('"')
                out2, err2 = sshh.ssh_command('echo $HOSTNAME') # Get computer name
                station_name = out2[0].strip().split(".")[0]
                new_rec = ix + ":" + serial_number[3] + ":" + station_name
                print(new_rec) # returns "192.168.9.1:DXXXXXXXXXX:MY_NAME"
                self.list_result.append(new_rec) # This will return None because writing to shared memory
            except:
                pass


    def thread_queue(self):
        dict_all = self.query_sort()
        for x in dict_all.values():
            t = threading.Thread(target=self.get_network_data, args=(x,))
            t.start()

Thanks.

change

                self.list_result.append(new_rec) # This will return None because writing to shared memory

to

                return new_rec

and work with that data outside of the function
In addition, you should handle your exception

Hi again,

thank you for your swift answer, i have updated code with what you said and result is same as before. You can see i have used list to append new values in loop which differs from what you wrote but it makes no difference result is 'None'.

Any other ideas where i am making mistake?

    def get_network_data(self, ixx):
        list_result = []
        for ix in ixx:
            if ix.startswith("10.172.9."):
                sshh = SshConnection(ix, "apple", "apple")
            else:
                sshh = SshConnection(ix, self.username, self.password)
            try:
                out1, err1 = sshh.ssh_command("ioreg -c IOPlatformExpertDevice "
                                              "-d 2 | awk '/IOPlatformSerialNumber/'")
                serial_number = out1[0].strip().split('"')
                out2, err2 = sshh.ssh_command('echo $HOSTNAME')
                station_name = out2[0].strip().split(".")[0]
                new_rec = ix + ":" + serial_number[3] + ":" + station_name
                # print(new_rec)
                list_result.append(new_rec)
                return list_result
            except (ssh_exception.AuthenticationException,
                    ssh_exception.NoValidConnectionsError,
                    ssh_exception.SSHException,
                    socket.timeout) as err:
                pass


    def thread_queue(self):
        dict_all = self.query_sort()
        for x in dict_all.values():
            t = threading.Thread(target=self.get_network_data, args=(x,))
            t.start()


new_nmap = NmapScan()
list_res = new_nmap.thread_queue() # >>>> Returns None
print(list_res)
for item in list_res:
    print(item) # >>>> TypeError: 'NoneType' object is not iterable

you load target with the return result, but you don't do anything with it.
How you process the data after you get it returned is up to you

Is it possible to write example how should i return list out of that function? I am not sure i understand what you mean.

First of all, since I see self in the argument list for your methods include self, I assume there is a class that this is operating from
(you should provide full listings or at least full classes when posting).
So under the class __init__ method, add a line self.target = None
then modify:

t = threading.Thread(target=self.get_network_data, args=(x,))
# to 
t = threading.Thread(self.target=self.get_network_data, args=(x,))

your data is now in self.target to do with as you please

Thank you for your response i have resolved it, it took me long enough to see the issue.
One thing still bothers me and it's speed of ssh over paramiko taking too long, when i am sending two separate requests with 3 sec timeouts. 400 computers 77 sec (I don't know if it is bad but i guess it's no good.

def get_network_data(self, ixx, qq):
        """Checking computers for Serial number and name"""
        self.set_result = set()
        qq.put(ixx)
        for ix in ixx:
            if ix.startswith("10.172.9."):
                sshh = SshConnection(ix, "apple", "apple")
            else:
                sshh = SshConnection(ix, self.username, self.password)
            try:
                out1, err1 = sshh.ssh_command("ioreg -c IOPlatformExpertDevice "
                                              "-d 2 | awk '/IOPlatformSerialNumber/'")
                serial_number = out1[0].strip().split('"')
                out2, err2 = sshh.ssh_command('echo $HOSTNAME')
                station_name = out2[0].strip().split(".")[0]
                new_rec = ix + ":" + serial_number[3] + ":" + station_name
                # print(new_rec)
                self.set_result.add(new_rec)
            except (ssh_exception.AuthenticationException,
                    ssh_exception.NoValidConnectionsError,
                    ssh_exception.SSHException,
                    socket.timeout) as err:
                pass
        qq.task_done()



    def thread_queue(self):
        qq = Queue()
        dict_all = self.query_sort()
        for x in dict_all.values():
            t = threading.Thread(self.target=self.get_network_data, args=(x, qq))
            t.start()
        t.join()
        qq.join()
        return self.set_result


new_nmap = NmapScan()
list_res = new_nmap.thread_queue() # >>>> Returns None