Python Forum
Python 2.7 Addition to dict is too slow
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python 2.7 Addition to dict is too slow
#1
I'm working on Python script. It takes information about subscribers' traffic from files and put it in special structures. And it works. But it works too slow. I've written the same algorith in PHP and it works much faster. I noticed Python spends a lot of time to put the data in dict. The PHP script spends 6 sec to process my test file, but the Python script - 12 sec (about 7 sec to get the data from the file and 5 sec to fill the structures). My structures look like this: struct[subscriberId][protocolId] = octents

And I use followed function to fill them:

def addBytesToStatStruct(struct, subscriberId, protocolId, octents):
  if subscriberId in struct:
    if protocolId in struct[subscriberId]:
      struct[subscriberId][protocolId] += octents
      return
      else:
        struct[subscriberId][protocolId] = octents
        return
  else:
    struct[subscriberId] = {protocolId : octents}
May be I do something wrong? I suppose my problem appears because of collisions happen during addition. As I know PHP uses chaining but Python uses open addressing. Could you give me a hint how can I make Python dict faster?
Reply


Messages In This Thread
Python 2.7 Addition to dict is too slow - by VolanD - May-03-2018, 04:57 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Multiply and Addition in the same loop statement with logic. joelraj 2 1,072 Feb-02-2023, 04:33 AM
Last Post: deanhystad
  forloop to compute sum by alternating from addition to subtraction JulianZ 3 1,870 Apr-02-2022, 09:36 AM
Last Post: DeaD_EyE
  My python code is running very slow on millions of records shantanu97 7 2,650 Dec-28-2021, 11:02 AM
Last Post: Larz60+
  String index out of bounds ( Python : Dict ) kommu 2 2,428 Jun-25-2020, 08:52 PM
Last Post: menator01
  Python 2 to 3 dict sorting joshuaprocious 2 63,140 May-14-2020, 03:28 PM
Last Post: joshuaprocious
  Sort a dict in dict cherry_cherry 4 80,826 Apr-08-2020, 12:25 PM
Last Post: perfringo
  Python list - group by dict key karthidec 2 9,470 Nov-25-2019, 06:58 AM
Last Post: buran
  addition for elements in lists of list ridgerunnersjw 3 3,134 Sep-15-2019, 07:11 AM
Last Post: perfringo
  multiplication by successive addition Zebrol 1 3,558 Sep-14-2019, 05:37 PM
Last Post: ichabod801
  Slow Python Code Jay123 3 2,526 Sep-09-2019, 08:46 AM
Last Post: Jay123

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020