Concatenate two dataframes

[moralear27]

Hi, I am currently trying to do which looks like a pretty complicated program. I have one dataframe without header which looks in this way:


Parameter1, Parameter2, Att1, Att1.value, Att2, Att2.value, Att3, Att3.value, Att4, Att4.value
Parameter1, Parameter2, Att2, Att2.value, Att3, Att3.value, Null, Null, Null, Null
Parameter1, Parameter2, Att1, Att1.value, Att3, Att3.value, Att2, Att2.value, Null, Null
Parameter1, Parameter2, Att3, Att3.value, Att1, Att1.value, Null, Null Null, Null
Parameter1, Parameter2, Att2, Att2.value, Att2, Att2.value, Null, Null Null, Null

(I have specified 10 colums for this dataframe to avoid issues)


Then I have created and empty dataframe just with headers:

Colum1, Colum2, Colum3, Colum4, Colum5


I need to concatenate both dataframes in the same way:

- Parameter1 and Parameter2 are fixed and value in every row should be placed in column1 and colum2.
- As you can see, attributes are a mess and they are not always placed in the same column in the first dataframe. Plus the attributes values are placed in the next column of each attribute. So I need to index Att1.value, Att2.value and Att3.value in colums 3, 4 and 5 (I am going to call these colums Att1, Att2 and Att3).
- The Att4 and Att4.value should be ignored and do not included in the new dataframe.


I would like to post a code attempt but to be honest, I do not even know how to start this. Do you think that it is possible to do? Sorry if it looks like if I would asking for a complete program but it is not the case, i just need to know if you think that it is possible to achieve and brief explanation about how would you do this. Thanks in advance.

[moralear27]

I am currently trying to evaluate every row in a dataframe to try to figure out if it has present some parameters and it works correctly:


import pandas as pd

d1 = {'1': ['1.1.1.1', '2.2.2.2'], '2': ['molear', 'laurylh'], '3': ["131", "132"], '4': ["3", "4"], '5': ["133", "131"], '6': ["3", "4"]}
df1 = pd.DataFrame(data=d1)

print (df1)


A=df1["1"]
B=df1["2"]
C="0"
D="0"


for i1 in df1.loc[0]:
if (i1=="131"):
C="resul1"

for i2 in df1.loc[0]:
if (i2=="133"):
D="result2"


d2 = {'IP': [A], 'User': [B], 'Att1': [C], 'Att2': [D]}
df2 = pd.DataFrame(data=d2)


df2.to_csv ('C:/Personal/result1.csv', index = False)


On the other hand, I am not sure how to say to my program to evaluate every row in df1 to check if it contains a specific value. ANy idea about how to do it? Thanks!

[moralear27]

I am close to achieve the result I am looking for. I have made a small dataframe just for testing purposes and with this code, the program is analyzing the df1 row per row and checking if the value 131 is present:

import pandas as pd

d1 = {'1': ['1.1.1.1', '2.2.2.2', '3.3.3.3'], '2': ['molear', 'laurylh', 'perea'], '3': ['131', '132', '4'], '4': ['3', '4', '6'], '5': ['133', '131', '56'], '6': ['3', "4", '8']}
df1 = pd.DataFrame(data=d1)

X=0

for index, row in df1.iterrows():
    for i1 in df1.loc[X]:
        if (i1=="131"): 
            C="Result"  
            X=X+1        

Now I am strugling to fill a new dataframe row by row wih the X result in each row according to related row in df1. Any idea would be welcome. thanks!