[Numpy] Load date/time from .txt to 'datetime64' type.

[water]

I have date/time data in .txt file likes below format:

1 2 3 4 5 6 7 8 9

2024-2-27, 0:0:0 2024-2-27, 3:7:2 2024-2-27, 4:11:3 2024-2-27, 6:3:5 2024-2-27, 13:10:6 2024-2-27, 20:20:7 2024-2-27, 21:30:9 2024-2-27, 22:40:10 2024-2-27, 23:50:12

I want to load that use numpy.loadtxt() and convert to 'datetime64' type, but 'datetime64' type require string format likes below:

1	numpy.array(['2020-02-27T15:23:07'], dtype = 'datetime64[s]')

so I must fill some '0', and concatenate date and time use 'T'?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

# fill '0' 2024-02-27, 00:00:00 2024-02-27, 03:07:02 2024-02-27, 04:11:03 2024-02-27, 06:03:05 2024-02-27, 13:10:06 2024-02-27, 20:20:07 2024-02-27, 21:30:09 2024-02-27, 22:40:10 2024-02-27, 23:50:12   # concatenate use 'T' 2024-02-27T00:00:00 2024-02-27T03:07:02 2024-02-27T04:11:03 2024-02-27T06:03:05 2024-02-27T13:10:06 2024-02-27T20:20:07 2024-02-27T21:30:09 2024-02-27T22:40:10 2024-02-27T23:50:12

Is it have simply way to do that?

[Gribouillis]

This may not be the fastest method but it seems to work

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

import datetime as dt import io import numpy as np   file = io.StringIO(     """\ 2024-2-27, 0:0:0 2024-2-27, 3:7:2 2024-2-27, 4:11:3 2024-2-27, 6:3:5 2024-2-27, 13:10:6 2024-2-27, 20:20:7 2024-2-27, 21:30:9 2024-2-27, 22:40:10 2024-2-27, 23:50:12 """ )     def gen(file):     for line in file:         t = dt.datetime.strptime(line, "%Y-%m-%d, %H:%M:%S\n")         yield t.isoformat()     a = np.loadtxt(gen(file), dtype="datetime64[s]") print(a)

Output:
['2024-02-27T00:00:00' '2024-02-27T03:07:02' '2024-02-27T04:11:03'
 '2024-02-27T06:03:05' '2024-02-27T13:10:06' '2024-02-27T20:20:07'
 '2024-02-27T21:30:09' '2024-02-27T22:40:10' '2024-02-27T23:50:12']

[Danishhafeez]

I have date/time data in .txt file likes below format:

1 2 3 4 5 6 7 8 9

2024-2-27, 0:0:0 2024-2-27, 3:7:2 2024-2-27, 4:11:3 2024-2-27, 6:3:5 2024-2-27, 13:10:6 2024-2-27, 20:20:7 2024-2-27, 21:30:9 2024-2-27, 22:40:10 2024-2-27, 23:50:12

I want to load that use numpy.loadtxt() and convert to 'datetime64' type, but 'datetime64' type require string format likes below:

1	numpy.array(['2020-02-27T15:23:07'], dtype = 'datetime64[s]')

so I must fill some '0', and concatenate date and time use 'T'?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

# fill '0' 2024-02-27, 00:00:00 2024-02-27, 03:07:02 2024-02-27, 04:11:03 2024-02-27, 06:03:05 2024-02-27, 13:10:06 2024-02-27, 20:20:07 2024-02-27, 21:30:09 2024-02-27, 22:40:10 2024-02-27, 23:50:12   # concatenate use 'T' 2024-02-27T00:00:00 2024-02-27T03:07:02 2024-02-27T04:11:03 2024-02-27T06:03:05 2024-02-27T13:10:06 2024-02-27T20:20:07 2024-02-27T21:30:09 2024-02-27T22:40:10 2024-02-27T23:50:12

Is it have simply way to do that?

You can achieve this by using the numpy.genfromtxt() function with a custom converter to parse your date/time data and convert it to datetime64 type.

import numpy as np

# Custom converter function to parse date/time strings and convert to datetime64
def datetime_converter(date_str, time_str):
datetime_str = f"{date_str.strip()}, {time_str.strip()}"
return np.datetime64(datetime_str)

# Load data from the text file using genfromtxt with the custom converter
data = np.genfromtxt('your_file.txt', delimiter=',', dtype='U', converters={0: lambda x: x, 1: lambda x: x}, unpack=True)

# Extract date and time columns
dates = data[0]
times = data[1]

# Convert to datetime64 using the custom converter
datetime_array = np.array([datetime_converter(date, time) for date, time in zip(dates, times)])

print(datetime_array)

i hope This code will load your data from the text file, parse the date and time strings, and convert them to datetime64 type using the custom converter function. The resulting datetime_array will contain the datetime values in the desired format.

Best regard
Danish hafeez | QA Assistant

buran write Mar-01-2024, 10:04 AM:
Please, use proper tags when post code, traceback, output, etc.
See BBcode help for more info.
Spam/advertisement link removed

[water]

def gen(file):
for line in file:
t = dt.datetime.strptime(line, "%Y-%m-%d, %H:%M:%S\n")
yield t.isoformat()

That seems a good method for this case, thanks.

[Gribouillis]

That seems a good method for this case, thanks.

You can skip a double conversion by creating the array with numpy.fromiter()

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

import datetime as dt import io import numpy as np   file = io.StringIO(     """\ 2024-2-27, 0:0:0 2024-2-27, 3:7:2 2024-2-27, 4:11:3 2024-2-27, 6:3:5 2024-2-27, 13:10:6 2024-2-27, 20:20:7 2024-2-27, 21:30:9 2024-2-27, 22:40:10 2024-2-27, 23:50:12 """ )     def parse(line):     return dt.datetime.strptime(line, "%Y-%m-%d, %H:%M:%S\n")     a = np.fromiter((parse(line) for line in file), dtype="datetime64[s]")   print(a)