Copy and append data from .dat file

[Makada]

Great, thanks alot for your help, much appreciated

---

I just tried as a test to remove the first data line in the sourcefile and close the running script and started again after some minutes, as soon as i started the script, an error is shown.
I cant run the script now.

Traceback (most recent call last):
  File "F:\Software\python definitief\copieer bron naar copie alleen nieuw data deze werkt.py", line 16, in <module>
    outfile.write(content)
ValueError: I/O operation on closed file.

The script i ran was this:

import time
import schedule    
starttime=time.time()



Input_file: "C:\\Campbellsci\\LoggerNet\\CR1000_Table1.dat"
Output_file: "C:\\Users\\Makada\\Desktop\\CR1000_Table1 - kopie.dat"

while True:
    """ Not to append duplicate data to output file"""
    existingLines = set(line.strip() for line in open("C:\\Users\\Makada\\Desktop\\CR1000_Table1 - kopie.dat"))
    outfile = open("C:\\Users\\Makada\\Desktop\\CR1000_Table1 - kopie.dat", "a+")
    for content in open("C:\\Campbellsci\\LoggerNet\\CR1000_Table1.dat", "r"):
     if content.strip() not in existingLines: # to void duplicate lines
        outfile.write(content)
        existingLines.add(content)
        outfile.close()

time.sleep(60 - time.time() % 60)

---

I removed the "outfile.close()" from the script, now it runs again.
Dont know if its working ok now...

---

I noticed double data figures and after a restart of the script (at 21:12:00) its showing data already shown, see a piece of output below:

"2020-01-17 21:08:00",2067,12.08,262,15.37,17.65,6.376,6.478,6.267,71.8867,71.96,71.82,1018.672,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.682,1.776,1.581,4.294,4.375,4.189,6.376,6.478,959,966,952,0,0,0,0,0,3.948,3.155
"2020-01-17 21:08:00",2067,12.08,262,15.37,17.65,6.376,6.478,6.267,71.8867,71.96,71.82,1018.672,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.682,1.776,1.581,4.294,4.375,4.189,6.376,6.478,959,966,952,0,0,0,0,0,3.948,3.155
"2020-01-17 21:09:00",2068,8.16,263.7,26.31,12.47,6.36,6.444,6.25,71.96032,72.01,71.91,1018.679,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.68,1.761,1.579,4.285,4.358,4.179,6.36,6.444,958,964,951,0,0,0,0,0,4.822,3.757
"2020-01-17 21:09:00",2068,8.16,263.7,26.31,12.47,6.36,6.444,6.25,71.96032,72.01,71.91,1018.679,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.68,1.761,1.579,4.285,4.358,4.179,6.36,6.444,958,964,951,0,0,0,0,0,4.822,3.757
"2020-01-17 21:10:00",2069,9.31,264.8,24.91,14,6.366,6.461,6.284,72.02,72.05,71.98,1018.684,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.697,1.794,1.613,4.295,4.374,4.213,6.366,6.461,958,965,953,0,0,0,0,0,4.497,3.601
"2020-01-17 21:11:00",2070,9.09,256.9,12.86,13.59,6.37,6.444,6.318,72.05154,72.09,72.03,1018.699,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.707,1.788,1.656,4.302,4.362,4.251,6.37,6.444,959,964,955,0,0,0,0,0,4.582,3.614
"2020-01-17 21:12:00",2071,9.87,259.8,23,14.7,6.371,6.461,6.258,72.08898,72.15,72.04,1018.698,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.716,1.804,1.61,4.305,4.378,4.198,6.371,6.461,959,965,951,0,0,0,0,0,4.378,3.516
"2020-01-17 21:10:00",2069,9.31,264.8,24.91,14,6.366,6.461,6.284,72.02,72.05,71.98,1018.684,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.697,1.794,1.613,4.295,4.374,4.213,6.366,6.461,958,965,953,0,0,0,0,0,4.497,3.601
"2020-01-17 21:11:00",2070,9.09,256.9,12.86,13.59,6.37,6.444,6.318,72.05154,72.09,72.03,1018.699,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.707,1.788,1.656,4.302,4.362,4.251,6.37,6.444,959,964,955,0,0,0,0,0,4.582,3.614
"2020-01-17 21:12:00",2071,9.87,259.8,23,14.7,6.371,6.461,6.258,72.08898,72.15,72.04,1018.698,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.716,1.804,1.61,4.305,4.378,4.198,6.371,6.461,959,965,951,0,0,0,0,0,4.378,3.516
"2020-01-17 21:13:00",2072,9.46,253.7,17.5,12.95,6.35,6.444,6.258,72.16513,72.23,72.11,1018.71,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.711,1.804,1.622,4.291,4.365,4.203,6.35,6.444,957,964,951,0,0,0,0,0,4.439,3.746
"2020-01-17 21:13:00",2072,9.46,253.7,17.5,12.95,6.35,6.444,6.258,72.16513,72.23,72.11,1018.71,1018.724,1018.662,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1.711,1.804,1.622,4.291,4.365,4.203,6.35,6.444,957,964,951,0,0,0,0,0,4.439,3.746