Pandas: how to split one row of data to multiple rows and columns in Python

Pandas: how to split one row of data to multiple rows and columns in Python - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Pandas: how to split one row of data to multiple rows and columns in Python (/thread-32598.html)

Pandas: how to split one row of data to multiple rows and columns in Python - GerardMoussendo - Feb-20-2021

I have a set of data with one row and several columns. I want to split it into multiple rows and 10 columns (kind of multiple dimensional). Here an example of my data( i have 1583717 samples in total):
VALUES: [ 0 0 0 ... 5740 -11760 8510]

Below is my code:

data = np.memmap("F:\data.pcm", dtype='h', mode='r')
print("VALUES:", data)
pylab.plot(data)
pylab.show()
np.save("data_signal.npy", data)
l = np.load("data_signal.npy")
print(l)

RE: Pandas: how to split one row of data to multiple rows and columns in Python - eddywinch82 - Feb-20-2021

Hi GerardMoussendo,

Could you post the Full Code ?

It would help me, with coming up with a solution for you.

Regards

Eddie Winch

RE: Pandas: how to split one row of data to multiple rows and columns in Python - perfringo - Feb-20-2021

I am not fully understand the problem. However, reshaping data in one row can be done as follows (data.csv is file containing numbers 0...99 in one row separated by commas):

import pandas as pd

arr = pd.read_csv('data.csv', header=None).values.reshape(10, 10)

arr will be:

Output:[[ 0  1  2  3  4  5  6  7  8  9]
 [10 11 12 13 14 15 16 17 18 19]
 [20 21 22 23 24 25 26 27 28 29]
 [30 31 32 33 34 35 36 37 38 39]
 [40 41 42 43 44 45 46 47 48 49]
 [50 51 52 53 54 55 56 57 58 59]
 [60 61 62 63 64 65 66 67 68 69]
 [70 71 72 73 74 75 76 77 78 79]
 [80 81 82 83 84 85 86 87 88 89]
 [90 91 92 93 94 95 96 97 98 99]]

RE: Pandas: how to split one row of data to multiple rows and columns in Python - GerardMoussendo - Feb-21-2021

(Feb-20-2021, 03:44 PM)eddywinch82 Wrote: Hi GerardMoussendo,

Could you post the Full Code ?

It would help me, with coming up with a solution for you.

Regards

Eddie Winch

Here is my entire code:

import numpy as np
import pylab
import matplotlib.pyplot as plt
import pandas as pd


data = np.memmap("F:\data.pcm", dtype='h', mode='r')
print("VALUES:", data)
pylab.plot(data)
pylab.show()
np.save("data_signal.npy", data)
l = np.load("data_signal.npy")
# print(l)
s = np.arange(len(l)).reshape(-1,5)
print(s)
pylab.plot(s)
pylab.show()

Now, it seems like

Quote:s = np.arange(len(l)).reshape(-1,5)

can work right? But how can I save the s reshaped data? I am having difficulty saving it in .npy
After reshaped, I have this result below:

Quote:[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[ 10 11 12 13 14]
...
[1583985 1583986 1583987 1583988 1583989]
[1583990 1583991 1583992 1583993 1583994]
[1583995 1583996 1583997 1583998 1583999]]

RE: Pandas: how to split one row of data to multiple rows and columns in Python - eddywinch82 - Feb-22-2021

Hi GerardMoussendo,

Try the following :-

np.savetxt("data.csv", s)

And if the numbers in the array, are separated by commas
for example, use the following line of Code :-

np.savetxt("data.csv", s, delimiter=",")

Or if the above lines of Code don't work.

Try :-

numpy.savetxt("data.csv", s)

Or :-

numpy.savetxt("data.csv", s, delimiter=",")

The delimiter doesn't have to be a comma, it could be something else.

The above lines of Code, saves to a text File, obviously, you can call
the .csv File whatever you like.

Or :-

pd.DataFrame(np_array).to_csv("path/to/file.csv")

Or if you don't want a header or index, use :-

to_csv("/path/to/file.csv", header=None, index=None)

I hope these lines of Code work for you.

If they don't work, could you upload the .pcm File, you get the Data from ?
And I could try the Code out myself, and get the answer you need ?

Regards

Eddie Winch