Python Forum

here is a script that shows the first 10 paths from os.walk() and the first 10 paths from the sort of paths from os.walk().

import os
from os.path import join
from sys import argv

argv.pop(0)
if not argv:
    exit('need one argument naming a file tree')

apex = argv.pop(0)
r = 64*'-'

paths = [apex]
for x,xd,xf in os.walk(apex):
    for d in xd:
        paths.append(join(x,d))
    for f in xf:
        paths.append(join(x,f))

names = paths[:]
names.sort()

for x in range(10):
    print(paths[x])
print(r,flush=True)

for x in range(10):
    print(names[x])
print(r,flush=True)

here is the result on my forums account:

Output:
t2a/forums /home/forums 5> py try_oswalk_sort.py /home/forums
/home/forums
/home/forums/requests
/home/forums/ijson
/home/forums/.subversion
/home/forums/animage
/home/forums/.screenshots
/home/forums/aws
/home/forums/futurist.se
/home/forums/Templates
/home/forums/.ssh-auth
----------------------------------------------------------------
/home/forums
/home/forums/.ICEauthority
/home/forums/.Xauthority
/home/forums/.alias
/home/forums/.alias.6204310359228514
/home/forums/.alias.6204310951603521
/home/forums/.audacity-data
/home/forums/.audacity-data/AutoSave
/home/forums/.audacity-data/Plug-Ins
/home/forums/.audacity-data/audacity.cfg
----------------------------------------------------------------

so, the problem is you incorrect expectation that os.walk() returns result in alphabetic/sorted order, which however is not the case

check
https://stackoverflow.com/a/18282401/4046632

Also, because paths has both dirs and files, it is normal to expect that when you sort it some [hidden] files/folders will come on the top, because of the . i.e. even if os.walk() returns folders in alphabetic order, when you sort mix of files and folders the result may differ.

i never said that os.walk() returns results in alphabetic/sorted order. i am trying to say that it does not. i am looking for tools that can use os.walk() and accomplish alphabetic/sorted order without loading the entire tree and doing sort on that much data (extreme memory demand in some cases).

the code i created does not use os.walk(). instead it uses os.listdir(). the result (generator yields) is in alphabetic/sorted order. it accomplishes this by sorting the results from os.listdir(). the generator can be given a specific sort key to do things like ignore leading non-alphanumeric characters or alphabetic case to produce a different order.

the results i get are correct. taking the results and doing a full tree sort (with special handling for the / directory separator to make it collate lower) gives the exact same order. i have a bash script i've been using for decades that does this by wrapping the POSIX/Unix sort command.