Appending lists into lists from function is not working for me

[BoredBannana]

Hey guys,

I'd like to preface, that I'm a complete beginner - this is actually my first bit of code outside of isolated lesson structures.

So I inted to build some sort of offline spotify clone for my music.

My file structure in the O:\DB folder looks like this

DB
Grimes / Art Angels / Song1, Song 2 ...
Interpol / Our Love to Admire / Song 1, Song 2...
Tool / Lateralus / Song1, Song 2...

Now the code is supposed to fetch data from the structure, so that I can neatly export the results into an sqlite library, with minimal manual labor. From there I'll create the connections via 1-n IDs.

artists = []
temp = []

def fetch(source):
    global artists, temp
    for entry in os.listdir(source):
        name = f"Name: {entry}"
        temp.append(name)
        folder_path = os.path.join(source, entry)
        temp.append(folder_path)
        artists.append(temp)
        # print(artists)
        temp.clear()

fetch(O:/DB)
print(artists)

The last line print(artists) hands out only empty lists like artists = [[], [], []]
wheras the comment print inside the loop does give me my desired outcome of
artists = [ [Name: Grimes, O:/DB/Grimes], [Name: Interpol, O:/DB/Interpol], [Name: Tool, O:/DB/Tool] ] .


Why is that ? And how do I get my funtion to produce my desired outcome? It works with temp.extend(), but that gives me only one list with all the data.

Also, if I remove the temp.clear() line, why do I get 3 x [ [Name: Grimes, O:/DB/Grimes, Name: Interpol, O:/DB/Interpol, Name: Tool, O:/DB/Tool ]
instead of
[Name: Grimes, O:/DB/Grimes]
[Name: Grimes, O:/DB/Grimes, Name: Interpol, O:/DB/Interpol]
[Name: Grimes, O:/DB/Grimes, Name: Interpol, O:/DB/Interpol, Name: Tool, O:/DB/Tool]

thanks in advance

[bowlofred]

I suspect a dict would be a better data structure than a list.
Passing "artists" explictly rather than using a global variable would be expected here.

Your actual problem is that you may be expecting list.append() to be copying the data when it does not. It just puts a reference to the object there. If the object changes, then the list it was appended to changes.

>>> temp = ["foo"]
>>> a = []
>>> a.append(temp)
>>> a
[['foo']]
>>> temp.clear()
>>> a
[[]]

So every time you clear temp, the data is gone. Get rid of temp entirely and this works and is easier to read.

import os

artists = []

def fetch(source):
    global artists
    for entry in os.listdir(source):
        name = f"Name: {entry}"
        artists.append([name, os.path.join(source, entry)])

fetch("O:/DB")
print(artists)

[deanhystad]

This creates an empty list object.
[]

This creates an empty list object and assigns a variable named "temp" to reference the list object.
temp= []

This adds the list object referenced by "temp" to a list referenced by a variable named "artists".
artists.append(temp)

If you execute that command multiple times in a loop, the list object referenced by "artists" will contain multiple references to the list object referenced by "temp". This may be easier to understand if we look at the at the actual objects.

artists = []
temp = []
artists.append(temp)
artists.append(temp)
artists.append(temp)
print(artists)
print("temp object", id(temp))
print("artist objects")
for thing in artists:
    print(id(thing))

Output:
temp object 2388589116672
artist objects
2388589116672
2388589116672
2388589116672

id(obj) returns the identifier for obj. From the output of the program you can see artists references the same object 3 times, and that object is the same object referenced by temp. If you change the object referenced by "temp", that change will be seen when you look at the objects referenced by "artists".

print(artists)
temp.append("A")
print(artists)

Output:
[[], [], []]
[['A'], ['A'], ['A']]

Now it should be obvious why you cannot reuse the same list object to add different information to the artists list. You need a different list object for each artist.

There is no need for this code in fetch()

global artists

fetch() references a the global variable "artists", but it never does anything that changes that variable. fetch() modifies the list referenced by "artists", but not the variable. The code works just fine without using global.

def fetch(source):
    for entry in os.listdir(source):
        artists.append([f"Name: {entry}", os.path.join(source, entry)])

Avoid using global variables in your programs, they lead to confusion as the number of global variables grows. I think your fetch() function should return the list of artists. Maybe like this:

from pathlib import Path

def fetch(source):
    entries = []
    source_path = Path(source)
    for file in source_path.iterdir():
        entries.append((f"Name:{file.name}", str(file.absolute())))
    return entries

artists = fetch("O:/DP")
print(artists)

I think the pathlib module is easier to use than os for scanning folders and manipulating filenames.

Python has a shorthand for making lists called a list comprehension. The above code written as a comprehension.

from pathlib import Path

def fetch(source):
    return [[f"Name:{file.name}", str(file.absolute())] for file in Path(source).iterdir()]

artists = fetch("O:/DP")
print(artists)

I agree with @bowlofred that it looks like you are trying to make something that kind of looks like a dictionary. There are dictionary comprehensions too!

from pathlib import Path

def fetch(source):
    return {file.name: str(file.absolute()) for file in Path(source).iterdir()}

artists = fetch("O:/DP")
print(artists)