Best way of taking a date prefix from a line and forming a file path from it?

Best way of taking a date prefix from a line and forming a file path from it? - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Best way of taking a date prefix from a line and forming a file path from it? (/thread-19962.html)

Best way of taking a date prefix from a line and forming a file path from it? - Skaperen - Jul-22-2019

i'm getting a string that begins with "YYYY-mm-dd HH:MM:SS". i need to get "YYYY-mm/dd/HH" from the string (to be the file name to write the string into). speed is most import for this as there can be bursts of hundreds per second (maybe even thousands, in rare cases). the code i am first thinking of is:

    name = s[:7]+'/'+s[8:10]+'/'+s[11:13]

f-strings is possible as this script is just for my use and i have 3.6.8, but i haven't learned their format syntax, yet. and since it is not for public use, pythonic is not a requirement.

RE: is there a better way? - snippsat - Jul-22-2019

Now is it more normal to use a date library when working with dates.

>>> from datetime import datetime
>>> 
>>> s = '2019-04-01 13:15:00'
>>> d = datetime.fromisoformat(s)
>>> d
datetime.datetime(2019, 4, 1, 13, 15)
>>> d.strftime('%Y-%m/%d/%H')
'2019-04/01/13'
>>> 
>>> # With f-string
>>> f'{d:%Y-%m/%d/%H}'
'2019-04/01/13'

Now is this format with date and hour together a little strange.
More like hour alone.

>>> f'{d:%Y-%m/%d %H}'
'2019-04/01 13'

Or replace with 00 if want that after hour.

>>> d
datetime.datetime(2019, 4, 1, 13, 15)
>>> h = d.replace(minute=00)
>>> f'{h:%Y-%m/%d %H:%M:%S}'
'2019-04/01 13:00:00'

RE: is there a better way? - Skaperen - Jul-22-2019

the string i get is already formatted. i just need to change a couple of characters. another option might be to spread the characters out into a list, update the two places that need to be changed, and join the characters back together. that, or maybe a bytearray.

RE: is there a better way? - Yoriz - Jul-22-2019

Your thread should have a more descriptive title of the problem at hand, actual examples of input and required output would also improve the chances of getting the answer you would like.

RE: is there a better way of reformatting a particular date string ? - DeaD_EyE - Jul-22-2019

In [13]: import datetime 
    ...:  
    ...:  
    ...: def manual_conv(s): 
    ...:     return s[:7]+'/'+s[8:10]+'/'+s[11:13] 
    ...:  
    ...:  
    ...: def manual_conv_fstring(s): 
    ...:     return f'{s[:7]}/{s[8:10]}{s[11:13]}' 
    ...:  
    ...: def dt_conv_fstring(s): 
    ...:     return f'{datetime.datetime.fromisoformat(s):%Y-%m/%d/%H}' 
    ...:  
    ...: def dt_conv(s): 
    ...:     return datetime.datetime.fromisoformat(s).strftime('%Y-%m/%d/%H') 
    ...:  
    ...:  
    ...: s = '2019-04-01 13:15:00' 
    ...: a, b, c, d = manual_conv(s), manual_conv(s), dt_conv(s), dt_conv_fstring(s) 
    ...: print(f'a: {a}, b: {b}, c: {c}') 
    ...: if a == b == c == d: 
    ...:     print('All functions return the same result') 
    ...:  
    ...: %timeit manual_conv(s) 
    ...: %timeit manual_conv_fstring(s) 
    ...: %timeit dt_conv(s) 
    ...: %timeit dt_conv_fstring(s)                                                                                        
a: 2019-04/01/13, b: 2019-04/01/13, c: 2019-04/01/13
All functions return the same result
438 ns ± 2.89 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
336 ns ± 7.46 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
2.24 µs ± 34.7 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
2.43 µs ± 14 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

RE: is there a better way of reformatting a particular date string ? - Skaperen - Jul-22-2019

here is what i have coded so far:

exe = argv.pop()
cmd = ['tcpdump','-lnttttU'] + argv

proc = Popen(cmd,stdout=PIPE)
pipe = proc.stdout

while True:
    buff = pipe.read()
    if not buff:
        pipe.close()
        proc.wait()
        proc = Popen(cmd,stdout=PIPE)
        pipe = proc.stdout
        continue
    prev_ymdh = None
    for line in buff.splitlines():
        ymdh = line[:7]+'/'+line[8:10]+'/'+line[11:13]
        if ymdh != prev_ymdh:
            curr_file.close()
            ymd = ymdh[:10]
            if ymd != prev_ymd:
                ym = ymd[:7]
                if ym != prev_ym:
                    os.mkdir(ym)
                    prev_ymd = None
                    prev_ym = ym
                os.mkdir(ymd)
                prev_ymdh = None
                prev_ymd = ymd
            curr_file = open(ymdh,'ab')
            prev_ymdh = ymdh
        curr_file.write(line)

i'm trying to improve line 17, right now. there could be other things wrong with this code that are not what i am asking for help on. and the imports are not coded, yet.

for example, i need to skip the close() the first time since no file is open, yet. if you see bugs, ignore them and let me find them. i may well end up rewriting most of it.