Python Forum
Best way of taking a date prefix from a line and forming a file path from it?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Best way of taking a date prefix from a line and forming a file path from it?
#1
i'm getting a string that begins with "YYYY-mm-dd HH:MM:SS". i need to get "YYYY-mm/dd/HH" from the string (to be the file name to write the string into). speed is most import for this as there can be bursts of hundreds per second (maybe even thousands, in rare cases). the code i am first thinking of is:
    name = s[:7]+'/'+s[8:10]+'/'+s[11:13]
f-strings is possible as this script is just for my use and i have 3.6.8, but i haven't learned their format syntax, yet. and since it is not for public use, pythonic is not a requirement.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
Now is it more normal to use a date library when working with dates.
>>> from datetime import datetime
>>> 
>>> s = '2019-04-01 13:15:00'
>>> d = datetime.fromisoformat(s)
>>> d
datetime.datetime(2019, 4, 1, 13, 15)
>>> d.strftime('%Y-%m/%d/%H')
'2019-04/01/13'
>>> 
>>> # With f-string
>>> f'{d:%Y-%m/%d/%H}'
'2019-04/01/13'
Now is this format with date and hour together a little strange.
More like hour alone.
>>> f'{d:%Y-%m/%d %H}'
'2019-04/01 13' 
Or replace with 00 if want that after hour.
>>> d
datetime.datetime(2019, 4, 1, 13, 15)
>>> h = d.replace(minute=00)
>>> f'{h:%Y-%m/%d %H:%M:%S}'
'2019-04/01 13:00:00'
Reply
#3
the string i get is already formatted. i just need to change a couple of characters. another option might be to spread the characters out into a list, update the two places that need to be changed, and join the characters back together. that, or maybe a bytearray.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#4
Your thread should have a more descriptive title of the problem at hand, actual examples of input and required output would also improve the chances of getting the answer you would like.
Reply
#5
In [13]: import datetime 
    ...:  
    ...:  
    ...: def manual_conv(s): 
    ...:     return s[:7]+'/'+s[8:10]+'/'+s[11:13] 
    ...:  
    ...:  
    ...: def manual_conv_fstring(s): 
    ...:     return f'{s[:7]}/{s[8:10]}{s[11:13]}' 
    ...:  
    ...: def dt_conv_fstring(s): 
    ...:     return f'{datetime.datetime.fromisoformat(s):%Y-%m/%d/%H}' 
    ...:  
    ...: def dt_conv(s): 
    ...:     return datetime.datetime.fromisoformat(s).strftime('%Y-%m/%d/%H') 
    ...:  
    ...:  
    ...: s = '2019-04-01 13:15:00' 
    ...: a, b, c, d = manual_conv(s), manual_conv(s), dt_conv(s), dt_conv_fstring(s) 
    ...: print(f'a: {a}, b: {b}, c: {c}') 
    ...: if a == b == c == d: 
    ...:     print('All functions return the same result') 
    ...:  
    ...: %timeit manual_conv(s) 
    ...: %timeit manual_conv_fstring(s) 
    ...: %timeit dt_conv(s) 
    ...: %timeit dt_conv_fstring(s)                                                                                        
a: 2019-04/01/13, b: 2019-04/01/13, c: 2019-04/01/13
All functions return the same result
438 ns ± 2.89 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
336 ns ± 7.46 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
2.24 µs ± 34.7 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
2.43 µs ± 14 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#6
here is what i have coded so far:

exe = argv.pop()
cmd = ['tcpdump','-lnttttU'] + argv

proc = Popen(cmd,stdout=PIPE)
pipe = proc.stdout

while True:
    buff = pipe.read()
    if not buff:
        pipe.close()
        proc.wait()
        proc = Popen(cmd,stdout=PIPE)
        pipe = proc.stdout
        continue
    prev_ymdh = None
    for line in buff.splitlines():
        ymdh = line[:7]+'/'+line[8:10]+'/'+line[11:13]
        if ymdh != prev_ymdh:
            curr_file.close()
            ymd = ymdh[:10]
            if ymd != prev_ymd:
                ym = ymd[:7]
                if ym != prev_ym:
                    os.mkdir(ym)
                    prev_ymd = None
                    prev_ym = ym
                os.mkdir(ymd)
                prev_ymdh = None
                prev_ymd = ymd
            curr_file = open(ymdh,'ab')
            prev_ymdh = ymdh
        curr_file.write(line)
i'm trying to improve line 17, right now. there could be other things wrong with this code that are not what i am asking for help on. and the imports are not coded, yet.

for example, i need to skip the close() the first time since no file is open, yet. if you see bugs, ignore them and let me find them. i may well end up rewriting most of it.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Compare current date on calendar with date format file name Fioravanti 1 112 Mar-26-2024, 08:23 AM
Last Post: Pedroski55
  Create dual folder on different path/drive based on the date agmoraojr 2 377 Jan-21-2024, 10:02 AM
Last Post: snippsat
  Python date format changes to date & time 1418 4 513 Jan-20-2024, 04:45 AM
Last Post: 1418
  Formatting a date time string read from a csv file DosAtPython 5 1,160 Jun-19-2023, 02:12 PM
Last Post: DosAtPython
  File path by adding various variables Mishal0488 2 964 Apr-28-2023, 07:17 PM
Last Post: deanhystad
  File "<string>", line 19, in <module> error is related to what? Frankduc 9 12,391 Mar-09-2023, 07:22 AM
Last Post: LocklearSusan
  Getting last line of each line occurrence in a file tester_V 1 811 Jan-31-2023, 09:29 PM
Last Post: deanhystad
  Script File Failure-Path Error? jerryf 13 3,313 Nov-30-2022, 09:58 AM
Last Post: jerryf
  prefix ID Number with 0,00 make 3 digit. mg24 1 708 Oct-06-2022, 07:20 AM
Last Post: ibreeden
  Writing string to file results in one character per line RB76SFJPsJJDu3bMnwYM 4 1,305 Sep-27-2022, 01:38 PM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020