Replace last 4 bytes of a timestamp

[mPlummers]

OK, I ran it.
It seems you are trying to do something very simple and making it very complicated.
Could you please tell me what the goal is,

starting value, amount of adjustment wanted, to (milliseconds?)
if you provide this, I can show hoe to simplify

It looks like you are trying to emulate 'C' using python.

Exactly, i'm quite sure this can be done easily with C, but my whole program is in Python and i'd like to try in this way. I have a timestamp in that form (readable string with year, month etc and with decimal seconds with 9 digits!). I have been told that, to find the next timestamp, i have to "replace the last 4 bytes with the ones i gave you". Now, i don't know if the bytes are referred to the timestamp or the related unix time. The only thing i know is that the two timestamps are very close to each other, order of milliseconds/seconds.
It seems a naive tecnique to me, i didn't find anything similar around the web.

Now i'm going to try the Nicolas code, let's see what happen!

---

Hello,

Can you try this?

from datetime import datetime
import math

old_date = "2019-08-06T07:37:51.358999986Z"
print("old date: {}".format(old_date))
splited_date = old_date.split('.')
old_date_without_milliseconds = splited_date[0]
old_date_milliseconds = splited_date[1][:-1]
print("old_date_without_milliseconds: {}".format(old_date_without_milliseconds))
print("old_date_milliseconds: {}".format(old_date_milliseconds))
new_bytes = [0xc8, 0xdc, 0xac, 0xa1]
t = datetime.strptime(old_date_without_milliseconds, "%Y-%m-%dT%H:%M:%S")
t = t.timestamp()
print("Time stamp from old_date_without_milliseconds: {}".format(t))
t = int(t) * 1000000000 + int(old_date_milliseconds)
print("Time stamp total * 1000000000: {}".format(t))
t = hex(t)
print("Time stamp total in hex: {}".format(t))
size_new_bytes = len(new_bytes) * 2
t = t[:-size_new_bytes]
for b in new_bytes:
    t += hex(b)[2:]
print("Time stamp in hex after substitution: {}".format(t))
t = int(t, 16)
print("Time stamp after substitution: {}".format(t))
partie_entiere = math.floor((t / 1000000000)) * 1000000000
decimals = t - partie_entiere
t = datetime.fromtimestamp(partie_entiere / 1000000000)
t = datetime.strftime(t, "%Y-%m-%dT%H:%M:%S.") + str(decimals) + "Z"
print("new date: {}".format(t))

Best Regards,

Nicolas TATARENKO

Hi Nicolas, thank you again. Now it seems to work, but i gave you the bytes in the inverse order. I put the in the opposite order and it works!

I'm curious to see what @Larz60+ would solve the problem. As i said, I only know that i have to replace the last 4 bytes of the timestamp with the ones they gave me. I'm quite sure there's a way to solve it without using unix time.

[Larz60+]

Exactly, i'm quite sure this can be done easily with C, but my whole program is in Python

I wasn't implying that you should use 'C', just that it looked like you were trying to emulate 'C'.

I have been told that, to find the next timestamp, i have to "replace the last 4 bytes with the ones i gave you".

I've never heard of such a thing. Where did you get such information?
timestamp is taken from the clock, trying to determine what the clock will return for a timestamp makes no sense, therefore, whoever told you this is taking you for a ride.

[mPlummers]

Hello,

Can you try this?

from datetime import datetime
import math

old_date = "2019-08-06T07:37:51.358999986Z"
print("old date: {}".format(old_date))
splited_date = old_date.split('.')
old_date_without_milliseconds = splited_date[0]
old_date_milliseconds = splited_date[1][:-1]
print("old_date_without_milliseconds: {}".format(old_date_without_milliseconds))
print("old_date_milliseconds: {}".format(old_date_milliseconds))
new_bytes = [0xc8, 0xdc, 0xac, 0xa1]
t = datetime.strptime(old_date_without_milliseconds, "%Y-%m-%dT%H:%M:%S")
t = t.timestamp()
print("Time stamp from old_date_without_milliseconds: {}".format(t))
t = int(t) * 1000000000 + int(old_date_milliseconds)
print("Time stamp total * 1000000000: {}".format(t))
t = hex(t)
print("Time stamp total in hex: {}".format(t))
size_new_bytes = len(new_bytes) * 2
t = t[:-size_new_bytes]
for b in new_bytes:
    t += hex(b)[2:]
print("Time stamp in hex after substitution: {}".format(t))
t = int(t, 16)
print("Time stamp after substitution: {}".format(t))
partie_entiere = math.floor((t / 1000000000)) * 1000000000
decimals = t - partie_entiere
t = datetime.fromtimestamp(partie_entiere / 1000000000)
t = datetime.strftime(t, "%Y-%m-%dT%H:%M:%S.") + str(decimals) + "Z"
print("new date: {}".format(t))

Best Regards,

Nicolas TATARENKO

Hi Nicolas,

i'm back, i'm still stucked on this problem. But now i have an additional information: the new timestamps should differ from the others by a time close to 1 second. It's not always exactly a second, but it's a good way to test if the code works. Do you think this info may be useful to find out the solution? Also, with your code, sometimes the result is a timestamp with 1973 year, instead of 2019.
Thank you again for your support

[Larz60+]

This will create a timestamp with current time (less fraction):

>>> import time
>>> def get_ts_filename(base, suffix):
...     return(f"{base}{int(time.time())}.{suffix}")
... 
>>> get_ts_filename('Jimbo', 'txt')
'Jimbo1568631640.txt'
>>>

[mPlummers]

Exactly, i'm quite sure this can be done easily with C, but my whole program is in Python

I wasn't implying that you should use 'C', just that it looked like you were trying to emulate 'C'.

I have been told that, to find the next timestamp, i have to "replace the last 4 bytes with the ones i gave you".

I've never heard of such a thing. Where did you get such information?
timestamp is taken from the clock, trying to determine what the clock will return for a timestamp makes no sense, therefore, whoever told you this is taking you for a ride.

This method is useful when you have to collect and send payloads with multiple datapoints taken at different times in a short period. In this case a single payload has ony one complete timestamp, and each datapoint has his own 4 bytes to replace to the payload timestamp. The timestamp is like a reference epoch. It's possible to do this operation in C/Java, but i can't see how can i do this in Python.

[avorane]

Hello,

When I try the code with new bytes that you hage given, i get a difference near 1 second (~1.1718)(WARNING: new bytes in the code are different of the question).

Can you give me the dates where the year is 1973, please?

Best Regards,

Nicolas TATARENKO

[Larz60+]

The timestamp is current time, so will change every second (every time called).
see: https://pymotw.com/3/time/

[mPlummers]

I found the solution using a more "Pythonic" method using Pandas:

aa= bytearray(new_bytes) 
temp = struct.unpack('<f', aa)
t = int(float(str(temp)[1:13])*1000000000)
rem_t = t/1000
rem_t = str(rem_t-int(rem_t))[1:]
rem_t = int(float(rem_t)*1000)

old_date = pd.Timestamp(old_date)
new_date = old_date - pd.Timedelta(t, 'ns') 

new_date = pd.datetime.strftime(new_date, "%Y-%m-%dT%H:%M:%S.%f")+str(rem_t)+"Z"

Thank you for all your support.

[Larz60+]

Yes indeed