Python Forum
created a pandas series instead of pandas DataFrame
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
created a pandas series instead of pandas DataFrame
#1
i created a pandas DataFrame for GPS satellite from a text file and i wanted to create a single DataFrame for distinct satellite, all goes as per my expectation but if for a given satellite there is only one observation it created a pandas Series instead of DataFrame. I want DataFrame because i need it to save it in HDF5 file.
Also i get a warning
FutureWarning: convert_objects is deprecated. To re-infer data dtypes for object columns, use DataFrame.infer_objects()
For all other conversions use the data-type specific converters pd.to_datetime, pd.to_timedelta and pd.to_numeric.


import pandas as pd
from textwrap import wrap
from datetime import datetime
def STF(a):
    '''STF takes input as a number string containing 'D' or 'E' and return the
    value as float and if the number string doesn't contain "D" or "E" it return 
    the original number in float.
    for example: 198701D-02=1987.01
                 198701E-02=1987.01 
                 198839=198839.00'''
    if 'D' in a:
        b = a.split("D")
        c = (float(b[0]))*(10**(int(b[1])))
        return(c)
    elif 'E' in a:
        b = a.split("E")
        c = (float(b[0]))*(10**(int(b[1])))
        return(c)
def gpsnav3df(filepath): 
    '''gpsnav3df takes input as filepath of Rinex3 navigation file of GPS  and
    give output in the form of dataframe containing all the information about the
    satellite like Satellite ID, Epoch, Clock Information etc.'''
    with open(filepath) as fp: #open file and returns the corresponding file object  
    
        data=fp.read() # read all the data into single string
        data1=data.split('END OF HEADER') # split a string into list
    
        #data section
        satdata=data1[1]
    
        satdata=satdata.strip() # remove the whitespaces from beginning and end of the string
        satdata=wrap(satdata,610) # split string into nth character of each word
    

        #creating Empty list    
        SatId ,Epoch,SV_clock_Bias,SV_clock_Drift,SV_clock_drift_Rate,IODE,Crs,Delta_n, M0, Cuc, e,Cus, sqrt_A ,Toe, Cic ,OMEGA_0, Cis, i0, Crc ,omega ,OMEGA_dot,IDOT,Codes ,GPS_Week ,L2P_data_Flag,SV_accuracy,SV_health, TGD, TODC ,Transmission_time , Fit_interval  = ([] for i in range(31) )
        
        for sat in satdata:
            #SV 
            satId = sat[0:3]  
            SatId.append(satId)
        
            #EPOCH
        
            time  = sat[4:23].replace(' ' , '')
            epoch = datetime.strptime(time,'%Y%m%d%H%M%S')
            Epoch.append(epoch)
    
            #SV_clock
        
            SV_clock_bias       = STF(sat[23:42]) # second
            SV_clock_Bias.append(SV_clock_bias)
        
            SV_clock_drift      = STF(sat[42:61]) #sec/sec
            SV_clock_Drift.append(SV_clock_drift)
                
            SV_clock_drift_rate = STF(sat[61:80]) # sec/sec2   
            SV_clock_drift_Rate.append(SV_clock_drift_rate)
            
            # Broadcast orbit-1
    
            IOD_E    =  STF(sat[85:104]) # Issue of Data,Ephemeris
            IODE.append(IOD_E)
            
            Cr_s     =  STF(sat[104:124]) #meters
            Crs.append(Cr_s)
            
            Delta_N =  STF(sat[124:142])# rad/sec
            Delta_n.append(Delta_N)
            
            
            M_0      =  STF(sat[142:161]) # radians
            M0.append(M_0)
            #broadcast orbit-2 
    
            Cu_c    = STF(sat[166:185]) #radians
            Cuc.append(Cu_c)
            
            e_      = STF(sat[185:204]) #Eccentricity
            e.append(e_)
            
            Cu_s    = STF(sat[204:223]) #radians
            Cus.append(Cu_s)
        
            sqrtA = STF(sat[223:242]) # sqrt(m)    
            sqrt_A.append(sqrtA)
    
            #broadcast orbit-3
    
            To_e     = STF(sat[247:266]) #Time of Ephemeris(sec of GPS week)
            Toe.append(To_e)
            
            Ci_c     = STF(sat[266:285]) #radians
            Cic.append(Ci_c)
            
            OMEGA0 = STF(sat[285:304]) #radians
            OMEGA_0.append(OMEGA0)
            
            Ci_s     = STF(sat[304:323]) #radians/sec
            Cis.append(Ci_s)
            
    
            #broadcast orbit-4
            
            i_0        = STF(sat[328:347]) #radian
            i0.append(i_0)
            
            Cr_c       = STF(sat[347:366]) #meters 
            Crc.append(Cr_c)
            
            omegA     = STF(sat[366:385]) #radians
            omega.append(omegA)
            
            OMEGA_Dot = STF(sat[385:404]) #radians/sec
            OMEGA_dot.append(OMEGA_Dot)
            
            #broadcast orbit-5
    
            IDO_T          = STF(sat[409:428])
            IDOT.append(IDO_T)
            
            codes         = STF(sat[428:447])
            Codes.append(codes)
            
            GPS_week      = STF(sat[447:466])
            GPS_Week.append(GPS_week)
            
            L2P_data_flag = STF(sat[466:485])
            L2P_data_Flag.append(L2P_data_flag)
        
        #broadcast orbit-6
        
            SV_Accuracy = STF(sat[490:509])
            SV_accuracy.append(SV_Accuracy)
            
            SV_Health   = STF(sat[509:528])
            SV_health.append(SV_Health)
            
            
            TG_D         = STF(sat[528:547])
            TGD.append(TG_D)
            
            IOD_C        = STF(sat[547:566])
            TODC.append(IOD_C)
        
        #broadcast orbit-7
            Transmission_Time = STF(sat[571:590])
            Transmission_time.append(Transmission_Time)
            
            Fit_Interval      = STF(sat[590:609])
            Fit_interval.append(Fit_Interval)
        
    df = pd.DataFrame(list(zip(SatId ,Epoch,SV_clock_Bias,SV_clock_Drift,SV_clock_drift_Rate,IODE,Crs,Delta_n, M0, Cuc, e,Cus, sqrt_A ,Toe, Cic ,OMEGA_0, Cis, i0, Crc ,omega ,OMEGA_dot,IDOT,Codes ,GPS_Week ,L2P_data_Flag,SV_accuracy,SV_health, TGD, TODC ,Transmission_time , Fit_interval )),
                      columns =['SatId' ,'Epoch','SV_clock_Bias','SV_clock_Drift','SV_clock_drift_Rate','IODE','Crs','Delta_n', 'M0', 'Cuc', 'e','Cus', 'sqrt_A' ,'Toe', 'Cic' ,'OMEGA_0', 'Cis',' i0',' Crc' ,'omega' ,'OMEGA_dot','IDOT','Codes' ,'GPS_Week' ,'L2P_data_Flag','SV_accuracy','SV_health', 'TGD', 'TODC' ,'Transmission_time' , 'Fit_interval']
                      ,dtype=object).convert_objects()
    
    df=df.set_index('SatId')
    
    return (df) 


filepath='E:/Project/RAW_Data/ABMF00GLP_R_20190201400_01H_GN.txt'
GPS=gpsnav3df(filepath)
G01=GPS.loc['G01']
print(G01)
print(type(G01))
G10=GPS.loc['G10']
print(G10)
print(type(G10))
Output:
Epoch 2019-01-20 06:00:00 SV_clock_Bias -0.000148412 SV_clock_Drift -6.48015e-12 SV_clock_drift_Rate 0 IODE 87 Crs -152.625 Delta_n 4.16696e-09 M0 1.87066 Cuc -7.94232e-06 e 0.00830994 Cus 7.9684e-06 sqrt_A 5153.67 Toe 21600 Cic -2.30968e-07 OMEGA_0 -0.722293 Cis 3.1665e-08 i0 0.974264 Crc 234.531 omega 0.687125 OMEGA_dot -7.9539e-09 IDOT -2.78583e-10 Codes 1 GPS_Week 2037 L2P_data_Flag 0 SV_accuracy 2 SV_health 0 TGD 5.58794e-09 TODC 87 Transmission_time 14400 Fit_interval None Name: G01, dtype: object <class 'pandas.core.series.Series'> Epoch SV_clock_Bias ... Transmission_time Fit_interval SatId ... G10 2019-01-20 13:59:44 0.000132 ... 47580.0 None G10 2019-01-20 16:00:00 0.000132 ... 50400.0 None [2 rows x 30 columns] <class 'pandas.core.frame.DataFrame'> E:/Project/code/n.py:172: FutureWarning: convert_objects is deprecated. To re-infer data dtypes for object columns, use DataFrame.infer_objects() For all other conversions use the data-type specific converters pd.to_datetime, pd.to_timedelta and pd.to_numeric.

Attached Files

.txt   ABMF00GLP_R_20190201400_01H_GN.txt (Size: 27.69 KB / Downloads: 0)
Reply


Messages In This Thread
created a pandas series instead of pandas DataFrame - by ibaad1406 - Aug-29-2019, 06:04 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Parsing and summing time deltas (duration) onto bar + pie charts using pandas - - DRY Drone4four 2 514 Feb-10-2024, 06:04 PM
Last Post: Drone4four
  Pandas hierarchical sum mariostg 1 379 Jan-26-2024, 03:47 PM
Last Post: mariostg
  Pandas keep existing format of Excel AlphaInc 2 881 Jan-11-2024, 03:44 AM
Last Post: plonkarchivist
  Grouping in pandas/multi-index data frame Aleqsie 3 607 Jan-06-2024, 03:55 PM
Last Post: deanhystad
  [pandas] TypeError: list indices must be integers or slices, not str but type is int. cspower 4 762 Dec-30-2023, 09:38 AM
Last Post: Gribouillis
  Assigning conditional values in Pandas Scott 3 725 Dec-19-2023, 03:10 AM
Last Post: Larz60+
  HTML Decoder pandas dataframe column mbrown009 3 962 Sep-29-2023, 05:56 PM
Last Post: deanhystad
  FutureWarning: Logical ops (and, or, xor) between Pandas objects and dtype-less seque NewBiee 5 1,490 Sep-12-2023, 03:15 PM
Last Post: deanhystad
  [solved] duplication in pandas BSDevo 0 523 Sep-06-2023, 10:47 PM
Last Post: BSDevo
  Help to control pandas.melt function vanphuht91 0 672 Jun-17-2023, 08:41 AM
Last Post: vanphuht91

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020