May-03-2018, 04:57 AM
I'm working on Python script. It takes information about subscribers' traffic from files and put it in special structures. And it works. But it works too slow. I've written the same algorith in PHP and it works much faster. I noticed Python spends a lot of time to put the data in dict. The PHP script spends 6 sec to process my test file, but the Python script - 12 sec (about 7 sec to get the data from the file and 5 sec to fill the structures). My structures look like this: struct[subscriberId][protocolId] = octents
And I use followed function to fill them:
And I use followed function to fill them:
def addBytesToStatStruct(struct, subscriberId, protocolId, octents): if subscriberId in struct: if protocolId in struct[subscriberId]: struct[subscriberId][protocolId] += octents return else: struct[subscriberId][protocolId] = octents return else: struct[subscriberId] = {protocolId : octents}May be I do something wrong? I suppose my problem appears because of collisions happen during addition. As I know PHP uses chaining but Python uses open addressing. Could you give me a hint how can I make Python dict faster?