Lists to strings

[millpond]

Fist, this is not a homework problem. But I'm noob enough for it to be in that category.
Here the code:

#!/usr/bin/python3

# Example of function returning multidimensional array, and turning it into normal string list. 

def changeme( mylist, zylist ):
    yulist = [mylist[2],450,700,666] # Function vaqriables
    xulist = [zylist[0],zylist[2],3,4] # to change argument vars
    return( yulist, xulist ) # returns 2 lists - 2 dimensional array
########## End of function  


### Function call
mylist=[0,0,0,0,0,0]
zylist=[1,1,1,1,1]
newlist = (changeme(mylist,zylist)) # [[1,2,3][4,5,6]] format of two arrays(lists)
###################3

###### Return processing 
print(newlist) #2 lists
print(newlist[0]) # 1st list
print(newlist[1]) # 2nd List 


numstring = ([str(y) for x in newlist for y in x]) # Converts num to string list in [1,2,3,...] order
print(numstring) 

mynewstr = (' '.join(numstring)) # removes commas, so string in [ 1 2 3 ...] order 
print(mynewstr)


################## Non-functioning 
# print(str(newlist).strip('[]'))
# print(str(newlist)[1:-1])

There are acxtually no problems with the code itself. Coming from Perl, I am a bit aghast at what looks like some unexpected complexity, however.

The idea of all basic data being arrays is a bit new to me.

So the question is if there is a simpler way, perhaps using modules, of converting lists, and list of lists (multidimensional arrays) into simple string lists without that for(each) and join command?

Most of the scripts I would like to convert from Perl have int and strings mixed together (as strings) and it was easy to parse whatever I wanted (=~ /regex/ ) . Looking for similar, even if as extensions (modules) to the language. Being new, not very familiar with the standard libraries here!

[ichabod801]

So the conversion is basically:

text = ' '.join([str(y) for x in newlist for y in x])

Is that what you think is too complicated? It doesn't seem that complicated to me. You could just do str(newlist), but that would give you a bunch of brackets and commas as well as the numbers.

The idea of all basic data being arrays is a bit new to me.

Uh, this is not true in Python. Basic data types in Python include ints and floats, which are not sequences or containers, and sets and dicts which are containers but not sequences. And there are lots of other basic data types that are less commonly used.

[millpond]

Not so much complicated, as counterintuitive to a n00b. In fairness my latest versions of Perl seem to require even more levels of obscurantism in dealing with Arrays of Arrays (AoA). Modules that worked before are giving problems now.
https://perldoc.perl.org/perldsc.html

But it seems that ITERTOOLS is what I am looking for
https://docs.python.org/3/library/itertools.html

I realize every language has data types, but some seem more 'duck typed' to certain types than others. Perl concentrates on string datatypes (think $_) but its array parsing (@_) complicates function arguments. Python seems more directed at simplifying array data (such as tuples imtead of pointers/references).

I definitely need to get used to the 'for...in' (vs foreach) syntax.

What I particularly enjoy about Python is the ease and simplicity of making basic internet connections and parsing data. Perl can aslo do it, but the scripts I have cobbled together for that purpose are excruciating, to say the least.

But as a n00b, only one question remains: is there a way to quickly search to see what method belongs to what module in the standard library (or Pypi)?
For example, I found a code snippet calling iterlist(), apparently from an import, but no module on Pypi with it specifically as a method PyDash and IterMark come up, but it does not have the specific function.

[ichabod801]

But as a n00b, only one question remains: is there a way to quickly search to see what method belongs to what module in the standard library (or Pypi)?

Not really. It's easy to do the other way around. If you import a module, dir(module_name) will give you all the names (classes/functions/constants) in it. Likewise, if you do dir() on an instance of a particular data type, it will show you the methods and attribute names for that data type. You can often web search for 'python method_name' for something in the standard library, which is well documented. Outside the standard library the documentation online is really hit or miss.

Good Python code should show you the import so you know where things are coming from. In fact, it's generally considered good form to import module instead of from module import name1, name2, name3, so the module will be directly referenced when the call is made. Some bigger modules are often abbreviated when imported, so you often see tk for tkinter and pd for pandas.

In terms of looping, definitely get familiar with the built-in functions enumerate and zip. You use enumerate to loop through a sequence and have the index of the current item, and you use zip to iterate over two or more lists at the same time. And, yes, itertools is a very useful module. Also check out collections, which provides more sequences and mappings.

[millpond]

Thank you for the info!

Oddly, dir(collections) is not found with idle either GUI or command line in Python 3.7.3 (Debian Buster, with Anaconda also installed) The site path is parsed afrter the system path, (which site installs appear to balk at overwriting).
PyPi doesnt recognize 'collections' but has helper modules for it, which I am installing, and will hopefully resolve that issie. Oddly, "import collections" raises no error in idle.

Perhaps if I explained at least part of the project I will be working with:
I have a library of over half a million files. The filenames have Title and Author\Artist and my first step is to parse them into categoriy directories, usually acrosss the network.

In Perl:
I parse the files into an Auth-Titl array.
I parse the Category directories into a CatDir array consisting of author/artist named directories of that subject (The script is done per subject).
And then I loop Auth-Title over CatDir, so that each media loops loops through all the subject authors for that categors, such as Sci-Fi.
This can take hours on a set of 10,000 files, with the Cat-Dir consisting of a few thousand dirs to match. When matched, the files are moved.

Perlmonks advise hashes(Dicts) to speed things up, but I havent figured out how to do this: Part of the problem is that nearly all author\artist names are repeated on the files, and titles(with subtitles) would be impractical as index keys. Perhaps an intermediate step to create Array of Arrays of some sort and use a numerical. Hence my interest in AoA's.

So the question here is: Is there a faster way to do loop-in-loop parsing with Python? There are numerous example files for basic operations, but I am concerend with loops with tens of thousands of itemms.

The question of a database comes up, but my experience with MySQL kind of makes me doubt that a DB can be of any use in parsing large amounts of data for file I/O in geological time spans.

Any suggestions, particularly as to modules for this scenario would be greatly appreciated :)

[ichabod801]

My first thought is that a dict would work. Have the key be the author name, and the value a list of titles. Or a list of tuples of (title, file_path, category, whatever).

My second thought is that if you are doing file operations you will want to look at the os and pathlib libraries.

My third thought is that I would expect the file transfers to be more of a bottle neck than a DB. There are DB modules for the typical SQL suspects and some others.

Note that dir(collections) won't work until you import collections. And learn to love the docs.