Python Forum
How can I make this function faster?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How can I make this function faster?
#1
Hi, I want to optimize this function and make it faster (if possible). Please let me know if there is a better way of doing this. Thanks!

# Reading data from a YML file.
def read(value):
    raw = open(file)
    reader = yaml.safe_load(raw)
    separate = value.split(".")
    i = 0
    x = reader.get(separate[i])
    for i in enumerate(separate):
        if isinstance(x, dict):
            try:
                num = (i[0] + 1)
                x = x.get(separate[num])
            except IndexError:
                return x
    return x
Reply
#2
with open(yaml_source) as yaml_file:
   reader = yaml.safe_load(yaml_file)
for key in value,split('.'):
    try:
        reader = reader[key]
    except (KeyError, TypeError):
        break
    return reader
i, x are terrible variable names - and your code is messy and inconsistent.
  • enumerate is redundant
  • You will never get IndexError with get
  • Use with operator to open file
  • PS When you get to a level that the value is not a dict - you may break out of the loop
Test everything in a Python shell (iPython, Azure Notebook, etc.)
  • Someone gave you an advice you liked? Test it - maybe the advice was actually bad.
  • Someone gave you an advice you think is bad? Test it before arguing - maybe it was good.
  • You posted a claim that something you did not test works? Be prepared to eat your hat.
Reply
#3
IndexError is not being thrown by the .get() statement. It's being thrown by separate[num]. I agree that the variable names are not good. I had already changed them shortly after making this post. with seems unnecessary if my current version works just fine. If you can show me that using with improves the speed of my function, then I will use it. As for your function, I tried implementing it and it simply did not work. Here is my edited version which still works.

# Reading data from a file.
def read(value):
    raw = open(file)
    reader = yaml.safe_load(raw)
    separate = value.split(".")
    original = reader.get(separate[0])
    for v, value in enumerate(separate):
        if isinstance(original, dict):
            try:
                original = original.get(separate[(v+1)])
            except IndexError:
                return original
    return original
Reply
#4
For one thing, you're testing original within the loop. I would think it would be faster to check original once, and only loop if it's a dict. Also, is it common or rare that it's not a dict? If it's common, and if statement is generally faster; if it's rare, a try/except block is generally faster. And (not familiar with yaml) what is it if it's not a dict? Could testing for that be faster? The same applies for your index error. I would guess not creating a separate variable fro num would be faster, but probably not significantly.
Craig "Ichabod" O'Brien - xenomind.com
I wish you happiness.
Recommended Tutorials: BBCode, functions, classes, text adventures
Reply
#5
(Jun-29-2018, 03:24 PM)Brennan Wrote: with seems unnecessary if my current version works just fine.

How about writing proper Python code?
You could have explained that you wanted to skip element 1
so
for value in separate[2:]:
instead of enumerate -and you'll eliminate try/except too.

Showing data example is usually a good idea too
Test everything in a Python shell (iPython, Azure Notebook, etc.)
  • Someone gave you an advice you liked? Test it - maybe the advice was actually bad.
  • Someone gave you an advice you think is bad? Test it before arguing - maybe it was good.
  • You posted a claim that something you did not test works? Be prepared to eat your hat.
Reply
#6
(Jun-29-2018, 04:05 PM)volcano63 Wrote: How about writing proper Python code?

How about not being an arrogant [CENSORED BY MOD]? If you have nothing nice to say, get off of my thread. I don't need people like you spreading negativity. I started coding Python two days ago. I'm still learning.

(Jun-29-2018, 03:30 PM)ichabod801 Wrote: For one thing, you're testing original within the loop. I would think it would be faster to check original once, and only loop if it's a dict. Also, is it common or rare that it's not a dict? If it's common, and if statement is generally faster; if it's rare, a try/except block is generally faster. And (not familiar with yaml) what is it if it's not a dict? Could testing for that be faster? The same applies for your index error. I would guess not creating a separate variable fro num would be faster, but probably not significantly.

Great questions. So, let me give some examples. Let's say we have a .YML with random data.

[Image: gjKcN3xGQ96g82lRIp2gCg.png]

If someone uses the read function as such: read("date") then our job is pretty simple. All we need to do is return yaml.safe_load(raw).get("date") However, let's say that I want to get python-version. I would need to do: yaml.safe_load(raw).get("settings.python-version"). Unfortunately, this does not work by default. So, how can we get around this? I begin by splitting their input: value.split(".") and assigning this array to the variable separate. We loop through the array. If the current looped element is not of type dict, then we are finished and we can return. Otherwise, we must append the function like such: yaml.safe_load(raw).get("settings").get("python-version") Using your advice, I changed the try/catch to an if-statement. This now makes the code look much cleaner. So thanks. :)

# Reading data from a file.
def read(value):
    with open(file) as raw:
        reader = yaml.safe_load(raw)

    separate = value.split(".")
    original = reader.get(separate[0])

    for v, value in enumerate(separate, 1):

        if v == len(separate):
            return original

        original = original.get(separate[v])

    return original
Reply
#7
Guys, chill out. Any more attitude or vulgarity and I am shutting this thread down.
Craig "Ichabod" O'Brien - xenomind.com
I wish you happiness.
Recommended Tutorials: BBCode, functions, classes, text adventures
Reply
#8
Sorry about that ichabod801. I'm now working on my edit function. It looks like this:

# Editing data of a file.
def edit(key, value):
    with open(file) as loader:
        document = yaml.safe_load(loader)
    document[key] = value
    with open(file, "w") as writer:
        yaml.dump(document, writer, default_flow_style=False)
It works fine if I do: edit("date", "some value"). However, if I want to do something like this: edit("settings.python-version", "some value") it doesn't. Any suggestions?
Reply
#9
How exactly does it not work? Are you getting an error? Is the output wrong (and how is it wrong)?
Craig "Ichabod" O'Brien - xenomind.com
I wish you happiness.
Recommended Tutorials: BBCode, functions, classes, text adventures
Reply
#10
The function wasn't setup to edit values inside of other values. For reference, the picture I posted above is the YML file that I am manipulating/reading. If I pass in date, it would look like this: document['date'] = value. This currently works perfectly.

To edit the python version, I would need to do: document['settings']['python-version'] = value. I can easily do this by hard-coding it. However, what if I want to take key (the function argument) which would be "settings.python-version" and convert it to: document['settings']['python-version']
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to make global list inside function CHANKC 6 2,979 Nov-26-2020, 08:05 AM
Last Post: CHANKC
  How to make this function general to create binary numbers? (many nested for loops) dospina 4 4,332 Jun-24-2020, 04:05 AM
Last Post: deanhystad
  how to make my function be a generator Skaperen 2 1,994 Jan-27-2020, 01:07 AM
Last Post: Skaperen
  How to make a function for auto generated id sunnyarora 2 2,872 May-03-2019, 12:47 PM
Last Post: Yoriz
  How can I make a faster search algorithm pianistseb 19 6,395 Apr-18-2019, 05:48 PM
Last Post: Larz60+
  To make an algorithm work faster pianistseb 3 2,757 Apr-01-2019, 08:42 AM
Last Post: Gribouillis
  Rewrite a function to make it work with 'bottle-pymysql' nikos 1 1,935 Feb-26-2019, 02:59 PM
Last Post: nikos
  Make a Function from Name String jge047 4 3,271 Dec-18-2017, 05:05 AM
Last Post: Terafy
  make a list of string in capital by using a function anancba 3 3,412 Nov-23-2017, 06:42 AM
Last Post: heiner55
  How do I make this code run faster? DontHurtMe 0 2,393 Nov-04-2017, 12:12 PM
Last Post: DontHurtMe

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020