Python Forum
Pandas: how to split one row of data to multiple rows and columns in Python
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Pandas: how to split one row of data to multiple rows and columns in Python
#1
I have a set of data with one row and several columns. I want to split it into multiple rows and 10 columns (kind of multiple dimensional). Here an example of my data( i have 1583717 samples in total):
VALUES: [ 0 0 0 ... 5740 -11760 8510]

Below is my code:

data = np.memmap("F:\data.pcm", dtype='h', mode='r')
print("VALUES:", data)
pylab.plot(data)
pylab.show()
np.save("data_signal.npy", data)
l = np.load("data_signal.npy")
print(l)
Reply
#2
Hi GerardMoussendo,

Could you post the Full Code ?

It would help me, with coming up with a solution for you.

Regards

Eddie Winch
GerardMoussendo likes this post
Reply
#3
I am not fully understand the problem. However, reshaping data in one row can be done as follows (data.csv is file containing numbers 0...99 in one row separated by commas):

import pandas as pd

arr = pd.read_csv('data.csv', header=None).values.reshape(10, 10)
arr will be:

Output:
[[ 0 1 2 3 4 5 6 7 8 9] [10 11 12 13 14 15 16 17 18 19] [20 21 22 23 24 25 26 27 28 29] [30 31 32 33 34 35 36 37 38 39] [40 41 42 43 44 45 46 47 48 49] [50 51 52 53 54 55 56 57 58 59] [60 61 62 63 64 65 66 67 68 69] [70 71 72 73 74 75 76 77 78 79] [80 81 82 83 84 85 86 87 88 89] [90 91 92 93 94 95 96 97 98 99]]
GerardMoussendo likes this post
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#4
(Feb-20-2021, 03:44 PM)eddywinch82 Wrote: Hi GerardMoussendo,

Could you post the Full Code ?

It would help me, with coming up with a solution for you.

Regards

Eddie Winch

Here is my entire code:

import numpy as np
import pylab
import matplotlib.pyplot as plt
import pandas as pd


data = np.memmap("F:\data.pcm", dtype='h', mode='r')
print("VALUES:", data)
pylab.plot(data)
pylab.show()
np.save("data_signal.npy", data)
l = np.load("data_signal.npy")
# print(l)
s = np.arange(len(l)).reshape(-1,5)
print(s)
pylab.plot(s)
pylab.show()
Now, it seems like
Quote:s = np.arange(len(l)).reshape(-1,5)
can work right? But how can I save the s reshaped data? I am having difficulty saving it in .npy
After reshaped, I have this result below:
Quote:[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[ 10 11 12 13 14]
...
[1583985 1583986 1583987 1583988 1583989]
[1583990 1583991 1583992 1583993 1583994]
[1583995 1583996 1583997 1583998 1583999]]
Reply
#5
Hi GerardMoussendo,

Try the following :-

np.savetxt("data.csv", s)
And if the numbers in the array, are separated by commas
for example, use the following line of Code :-

np.savetxt("data.csv", s, delimiter=",")
Or if the above lines of Code don't work.

Try :-

numpy.savetxt("data.csv", s)
Or :-

numpy.savetxt("data.csv", s, delimiter=",")
The delimiter doesn't have to be a comma, it could be something else.

The above lines of Code, saves to a text File, obviously, you can call
the .csv File whatever you like.

Or :-

pd.DataFrame(np_array).to_csv("path/to/file.csv")
Or if you don't want a header or index, use :-

to_csv("/path/to/file.csv", header=None, index=None)
I hope these lines of Code work for you.

If they don't work, could you upload the .pcm File, you get the Data from ?
And I could try the Code out myself, and get the answer you need ?

Regards

Eddie Winch
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How is pandas modifying all rows in an assignment - python-newbie question markm74 1 653 Nov-28-2023, 10:36 PM
Last Post: deanhystad
  python convert multiple files to multiple lists MCL169 6 1,436 Nov-25-2023, 05:31 AM
Last Post: Iqratech
  Better python library to create ER Diagram by using pandas data frames as tables klllmmm 0 989 Oct-19-2023, 01:01 PM
Last Post: klllmmm
  Using pyodbc&pandas to load a Table data to df tester_V 3 747 Sep-09-2023, 08:55 PM
Last Post: tester_V
  how do you style data frame that has empty rows. gsaray101 0 498 Sep-08-2023, 05:20 PM
Last Post: gsaray101
  How to check multiple columns value within range SamLiu 2 1,103 Mar-13-2023, 09:32 AM
Last Post: SamLiu
  Converting a json file to a dataframe with rows and columns eyavuz21 13 4,169 Jan-29-2023, 03:59 PM
Last Post: eyavuz21
  (Python) Pulling data from UA Google Analytics with more than 100k rows into csv. Stockers 0 1,170 Dec-19-2022, 11:11 PM
Last Post: Stockers
  Extracting Data into Columns using pdfplumber arvin 17 5,199 Dec-17-2022, 11:59 AM
Last Post: arvin
  How to properly format rows and columns in excel data from parsed .txt blocks jh67 7 1,802 Dec-12-2022, 08:22 PM
Last Post: jh67

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020