Python Forum
Pandas: how to split one row of data to multiple rows and columns in Python
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Pandas: how to split one row of data to multiple rows and columns in Python
#1
I have a set of data with one row and several columns. I want to split it into multiple rows and 10 columns (kind of multiple dimensional). Here an example of my data( i have 1583717 samples in total):
VALUES: [ 0 0 0 ... 5740 -11760 8510]

Below is my code:

data = np.memmap("F:\data.pcm", dtype='h', mode='r')
print("VALUES:", data)
pylab.plot(data)
pylab.show()
np.save("data_signal.npy", data)
l = np.load("data_signal.npy")
print(l)
Reply
#2
Hi GerardMoussendo,

Could you post the Full Code ?

It would help me, with coming up with a solution for you.

Regards

Eddie Winch
GerardMoussendo likes this post
Reply
#3
I am not fully understand the problem. However, reshaping data in one row can be done as follows (data.csv is file containing numbers 0...99 in one row separated by commas):

import pandas as pd

arr = pd.read_csv('data.csv', header=None).values.reshape(10, 10)
arr will be:

Output:
[[ 0 1 2 3 4 5 6 7 8 9] [10 11 12 13 14 15 16 17 18 19] [20 21 22 23 24 25 26 27 28 29] [30 31 32 33 34 35 36 37 38 39] [40 41 42 43 44 45 46 47 48 49] [50 51 52 53 54 55 56 57 58 59] [60 61 62 63 64 65 66 67 68 69] [70 71 72 73 74 75 76 77 78 79] [80 81 82 83 84 85 86 87 88 89] [90 91 92 93 94 95 96 97 98 99]]
GerardMoussendo likes this post
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#4
(Feb-20-2021, 03:44 PM)eddywinch82 Wrote: Hi GerardMoussendo,

Could you post the Full Code ?

It would help me, with coming up with a solution for you.

Regards

Eddie Winch

Here is my entire code:

import numpy as np
import pylab
import matplotlib.pyplot as plt
import pandas as pd


data = np.memmap("F:\data.pcm", dtype='h', mode='r')
print("VALUES:", data)
pylab.plot(data)
pylab.show()
np.save("data_signal.npy", data)
l = np.load("data_signal.npy")
# print(l)
s = np.arange(len(l)).reshape(-1,5)
print(s)
pylab.plot(s)
pylab.show()
Now, it seems like
Quote:s = np.arange(len(l)).reshape(-1,5)
can work right? But how can I save the s reshaped data? I am having difficulty saving it in .npy
After reshaped, I have this result below:
Quote:[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[ 10 11 12 13 14]
...
[1583985 1583986 1583987 1583988 1583989]
[1583990 1583991 1583992 1583993 1583994]
[1583995 1583996 1583997 1583998 1583999]]
Reply
#5
Hi GerardMoussendo,

Try the following :-

np.savetxt("data.csv", s)
And if the numbers in the array, are separated by commas
for example, use the following line of Code :-

np.savetxt("data.csv", s, delimiter=",")
Or if the above lines of Code don't work.

Try :-

numpy.savetxt("data.csv", s)
Or :-

numpy.savetxt("data.csv", s, delimiter=",")
The delimiter doesn't have to be a comma, it could be something else.

The above lines of Code, saves to a text File, obviously, you can call
the .csv File whatever you like.

Or :-

pd.DataFrame(np_array).to_csv("path/to/file.csv")
Or if you don't want a header or index, use :-

to_csv("/path/to/file.csv", header=None, index=None)
I hope these lines of Code work for you.

If they don't work, could you upload the .pcm File, you get the Data from ?
And I could try the Code out myself, and get the answer you need ?

Regards

Eddie Winch
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  [Solved] Using readlines to read data file and sum columns Laplace12 4 169 Jun-16-2021, 12:46 PM
Last Post: Laplace12
  Pandas DataFrame combine rows by column value, where Date Rows are NULL rhat398 0 240 May-04-2021, 10:51 PM
Last Post: rhat398
  Load the data from multiple source files to one table amy83 2 286 Apr-27-2021, 12:33 AM
Last Post: Pedroski55
Star Split and organize my Pandas Dataframe brunolelli 4 412 Apr-18-2021, 03:00 AM
Last Post: brunolelli
  API Gateway to manage multiple API's get put data robsuttonjr 2 510 Mar-09-2021, 04:09 PM
Last Post: robsuttonjr
  Indexing [::-1] to Reverse ALL 2D Array Rows, ALL 3D, 4D Array Columns & Rows Python Jeremy7 8 1,154 Mar-02-2021, 01:54 AM
Last Post: Jeremy7
  Yahoo_fin, Pandas: how to convert data table structure in csv file detlefschmitt 14 1,546 Feb-15-2021, 12:58 PM
Last Post: detlefschmitt
  How to filter out Column data From Multiple rows data? firaki12345 10 937 Feb-06-2021, 04:54 AM
Last Post: buran
  Inserting multiple rows in a single request. swaroop 2 484 Jan-11-2021, 01:34 PM
Last Post: swaroop
  Fetching data from multiple tables in a single request. swaroop 0 326 Jan-09-2021, 04:23 PM
Last Post: swaroop

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020