Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 created a pandas series instead of pandas DataFrame
#1
i created a pandas DataFrame for GPS satellite from a text file and i wanted to create a single DataFrame for distinct satellite, all goes as per my expectation but if for a given satellite there is only one observation it created a pandas Series instead of DataFrame. I want DataFrame because i need it to save it in HDF5 file.
Also i get a warning
FutureWarning: convert_objects is deprecated. To re-infer data dtypes for object columns, use DataFrame.infer_objects()
For all other conversions use the data-type specific converters pd.to_datetime, pd.to_timedelta and pd.to_numeric.


import pandas as pd
from textwrap import wrap
from datetime import datetime
def STF(a):
    '''STF takes input as a number string containing 'D' or 'E' and return the
    value as float and if the number string doesn't contain "D" or "E" it return 
    the original number in float.
    for example: 198701D-02=1987.01
                 198701E-02=1987.01 
                 198839=198839.00'''
    if 'D' in a:
        b = a.split("D")
        c = (float(b[0]))*(10**(int(b[1])))
        return(c)
    elif 'E' in a:
        b = a.split("E")
        c = (float(b[0]))*(10**(int(b[1])))
        return(c)
def gpsnav3df(filepath): 
    '''gpsnav3df takes input as filepath of Rinex3 navigation file of GPS  and
    give output in the form of dataframe containing all the information about the
    satellite like Satellite ID, Epoch, Clock Information etc.'''
    with open(filepath) as fp: #open file and returns the corresponding file object  
    
        data=fp.read() # read all the data into single string
        data1=data.split('END OF HEADER') # split a string into list
    
        #data section
        satdata=data1[1]
    
        satdata=satdata.strip() # remove the whitespaces from beginning and end of the string
        satdata=wrap(satdata,610) # split string into nth character of each word
    

        #creating Empty list    
        SatId ,Epoch,SV_clock_Bias,SV_clock_Drift,SV_clock_drift_Rate,IODE,Crs,Delta_n, M0, Cuc, e,Cus, sqrt_A ,Toe, Cic ,OMEGA_0, Cis, i0, Crc ,omega ,OMEGA_dot,IDOT,Codes ,GPS_Week ,L2P_data_Flag,SV_accuracy,SV_health, TGD, TODC ,Transmission_time , Fit_interval  = ([] for i in range(31) )
        
        for sat in satdata:
            #SV 
            satId = sat[0:3]  
            SatId.append(satId)
        
            #EPOCH
        
            time  = sat[4:23].replace(' ' , '')
            epoch = datetime.strptime(time,'%Y%m%d%H%M%S')
            Epoch.append(epoch)
    
            #SV_clock
        
            SV_clock_bias       = STF(sat[23:42]) # second
            SV_clock_Bias.append(SV_clock_bias)
        
            SV_clock_drift      = STF(sat[42:61]) #sec/sec
            SV_clock_Drift.append(SV_clock_drift)
                
            SV_clock_drift_rate = STF(sat[61:80]) # sec/sec2   
            SV_clock_drift_Rate.append(SV_clock_drift_rate)
            
            # Broadcast orbit-1
    
            IOD_E    =  STF(sat[85:104]) # Issue of Data,Ephemeris
            IODE.append(IOD_E)
            
            Cr_s     =  STF(sat[104:124]) #meters
            Crs.append(Cr_s)
            
            Delta_N =  STF(sat[124:142])# rad/sec
            Delta_n.append(Delta_N)
            
            
            M_0      =  STF(sat[142:161]) # radians
            M0.append(M_0)
            #broadcast orbit-2 
    
            Cu_c    = STF(sat[166:185]) #radians
            Cuc.append(Cu_c)
            
            e_      = STF(sat[185:204]) #Eccentricity
            e.append(e_)
            
            Cu_s    = STF(sat[204:223]) #radians
            Cus.append(Cu_s)
        
            sqrtA = STF(sat[223:242]) # sqrt(m)    
            sqrt_A.append(sqrtA)
    
            #broadcast orbit-3
    
            To_e     = STF(sat[247:266]) #Time of Ephemeris(sec of GPS week)
            Toe.append(To_e)
            
            Ci_c     = STF(sat[266:285]) #radians
            Cic.append(Ci_c)
            
            OMEGA0 = STF(sat[285:304]) #radians
            OMEGA_0.append(OMEGA0)
            
            Ci_s     = STF(sat[304:323]) #radians/sec
            Cis.append(Ci_s)
            
    
            #broadcast orbit-4
            
            i_0        = STF(sat[328:347]) #radian
            i0.append(i_0)
            
            Cr_c       = STF(sat[347:366]) #meters 
            Crc.append(Cr_c)
            
            omegA     = STF(sat[366:385]) #radians
            omega.append(omegA)
            
            OMEGA_Dot = STF(sat[385:404]) #radians/sec
            OMEGA_dot.append(OMEGA_Dot)
            
            #broadcast orbit-5
    
            IDO_T          = STF(sat[409:428])
            IDOT.append(IDO_T)
            
            codes         = STF(sat[428:447])
            Codes.append(codes)
            
            GPS_week      = STF(sat[447:466])
            GPS_Week.append(GPS_week)
            
            L2P_data_flag = STF(sat[466:485])
            L2P_data_Flag.append(L2P_data_flag)
        
        #broadcast orbit-6
        
            SV_Accuracy = STF(sat[490:509])
            SV_accuracy.append(SV_Accuracy)
            
            SV_Health   = STF(sat[509:528])
            SV_health.append(SV_Health)
            
            
            TG_D         = STF(sat[528:547])
            TGD.append(TG_D)
            
            IOD_C        = STF(sat[547:566])
            TODC.append(IOD_C)
        
        #broadcast orbit-7
            Transmission_Time = STF(sat[571:590])
            Transmission_time.append(Transmission_Time)
            
            Fit_Interval      = STF(sat[590:609])
            Fit_interval.append(Fit_Interval)
        
    df = pd.DataFrame(list(zip(SatId ,Epoch,SV_clock_Bias,SV_clock_Drift,SV_clock_drift_Rate,IODE,Crs,Delta_n, M0, Cuc, e,Cus, sqrt_A ,Toe, Cic ,OMEGA_0, Cis, i0, Crc ,omega ,OMEGA_dot,IDOT,Codes ,GPS_Week ,L2P_data_Flag,SV_accuracy,SV_health, TGD, TODC ,Transmission_time , Fit_interval )),
                      columns =['SatId' ,'Epoch','SV_clock_Bias','SV_clock_Drift','SV_clock_drift_Rate','IODE','Crs','Delta_n', 'M0', 'Cuc', 'e','Cus', 'sqrt_A' ,'Toe', 'Cic' ,'OMEGA_0', 'Cis',' i0',' Crc' ,'omega' ,'OMEGA_dot','IDOT','Codes' ,'GPS_Week' ,'L2P_data_Flag','SV_accuracy','SV_health', 'TGD', 'TODC' ,'Transmission_time' , 'Fit_interval']
                      ,dtype=object).convert_objects()
    
    df=df.set_index('SatId')
    
    return (df) 


filepath='E:/Project/RAW_Data/ABMF00GLP_R_20190201400_01H_GN.txt'
GPS=gpsnav3df(filepath)
G01=GPS.loc['G01']
print(G01)
print(type(G01))
G10=GPS.loc['G10']
print(G10)
print(type(G10))
Output:
Epoch 2019-01-20 06:00:00 SV_clock_Bias -0.000148412 SV_clock_Drift -6.48015e-12 SV_clock_drift_Rate 0 IODE 87 Crs -152.625 Delta_n 4.16696e-09 M0 1.87066 Cuc -7.94232e-06 e 0.00830994 Cus 7.9684e-06 sqrt_A 5153.67 Toe 21600 Cic -2.30968e-07 OMEGA_0 -0.722293 Cis 3.1665e-08 i0 0.974264 Crc 234.531 omega 0.687125 OMEGA_dot -7.9539e-09 IDOT -2.78583e-10 Codes 1 GPS_Week 2037 L2P_data_Flag 0 SV_accuracy 2 SV_health 0 TGD 5.58794e-09 TODC 87 Transmission_time 14400 Fit_interval None Name: G01, dtype: object <class 'pandas.core.series.Series'> Epoch SV_clock_Bias ... Transmission_time Fit_interval SatId ... G10 2019-01-20 13:59:44 0.000132 ... 47580.0 None G10 2019-01-20 16:00:00 0.000132 ... 50400.0 None [2 rows x 30 columns] <class 'pandas.core.frame.DataFrame'> E:/Project/code/n.py:172: FutureWarning: convert_objects is deprecated. To re-infer data dtypes for object columns, use DataFrame.infer_objects() For all other conversions use the data-type specific converters pd.to_datetime, pd.to_timedelta and pd.to_numeric.


Attached Files
.txt   ABMF00GLP_R_20190201400_01H_GN.txt (Size: 27.69 KB / Downloads: 0)
Quote
#2
Docstring of first function states “STF takes input as a number string containing 'D' or 'E' and return the value as float and if the number string doesn't contain "D" or "E" it return the original number in float.”

I don’t observe returning original number as float.
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Quote
#3
@perfringo:
yeah, you are right i changed it but the main problem of getting series instead of DataFrame is still there.
Quote
#4
Maybe that helps Convert Series to DataFrame.
Quote
#5
Or you can try call .loc correctly:

>>> import pandas as pd
>>> df = pd.DataFrame({'chars': (*'abc',), 'nums': (*range(3),)})
>>> df
 chars  nums
0     a     0
1     b     1
2     c     2
pandas.core.frame.DataFrame
>>> type(df.loc[0])
<class 'pandas.core.series.Series'>
>>> type(df.loc[[0]])                  # note additional brackets
<class 'pandas.core.frame.DataFrame'>
This and some other examples available in pandas documenation: pandas.DataFrame.loc
ibaad1406 likes this post
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Quote
#6
Keeping blinders on to your code and just answering the question posed, I am with ThomasL but this link Series.to_frame gives more detail and some examples. Hope this helps.
Quote
#7
@perfringo:
thanks, it works
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  strange error from pandas dataframe djf123 1 290 Jul-27-2020, 05:25 AM
Last Post: scidam
  Creating many csv files from Pandas EMA 0 161 Jul-26-2020, 06:39 PM
Last Post: EMA
  [pandas] How to reshape the list Mekala 6 253 Jul-26-2020, 12:49 AM
Last Post: Mekala
  How to compare two json and write to third json differences with pandas and numpy onenessboy 0 200 Jul-24-2020, 01:56 PM
Last Post: onenessboy
  problem with opening csv file in pandas saratha 2 254 Jul-21-2020, 10:55 AM
Last Post: saratha
  Python dict to pandas df waleed3011 3 173 Jul-18-2020, 04:08 AM
Last Post: scidam
  Pandas DataFrame not updating HelpMePlease 3 276 Jul-11-2020, 07:19 PM
Last Post: jefsummers
  Pandas DataFrame visual Truman 8 346 Jul-10-2020, 06:11 AM
Last Post: hussainmujtaba
  pandas read_csv can't handle missing data mrdominikku 0 238 Jul-09-2020, 12:26 PM
Last Post: mrdominikku
  pandas.Series IoannisDem 2 233 Jul-09-2020, 07:14 AM
Last Post: IoannisDem

Forum Jump:


Users browsing this thread: 1 Guest(s)