Python Forum
How do I convert my data so it works with Pandas?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How do I convert my data so it works with Pandas?
#1
I was thinking of looking at linear regression for a set of error data. Then, be able to make a prediction what the error count might be in some future week.

These data have two columns:

(1) "ErrorDate" -> week number (of year), and
(2) "ErrorCount" (how many errors did the system have in that week).

I would imagine these data are pretty noisy (random), but who knows?

Anyway, I tried to load this data and do a basic LinearRegression fit test with Panda and scikit-Learn but got an error.

ERROR: "ValueError: Expected 2D array, got 1D array instead:"


--
The code seems so simple, like it should work:
# Read CSV data into dataframe

thedf = pd.read_csv("Errors.csv", sep=",") # Read 2 column data into Pandas DataFrame

X_train, X_test, y_train, y_test = train_test_split(
thedf['ErrorCount'], thedf['ErrorDate'], random_state=0)

print (ussdf.head())

>>>> Prints:
ErrorDate ErrorCount
0 1 80
1 2 118
2 3 249
3 4 397
4 5 159

So far, so good..

But, the shape is apparently wrong and I get the error noted above.

print("X_test shape: {}".format(X_test.shape))
print("y_test shape: {}".format(y_test.shape))

>>>> Prints:
X_test shape: (13,)
y_test shape: (13,)

--

So, I see the shape is the problem, but it's not clear to me from searches I did how to change it. This is probably a super simple question. I have a Pandas book on order but it won't arrive for another week.

Suggestions?

Thanks very much in advance,
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Parsing "aTimeLogger" Android app data to graphs using pandas Drone4four 8 3,263 Jun-23-2024, 07:12 AM
Last Post: Drone4four
  Grouping in pandas/multi-index data frame Aleqsie 3 2,303 Jan-06-2024, 03:55 PM
Last Post: deanhystad
Smile How to further boost the data read write speed using pandas tjk9501 1 2,041 Nov-14-2022, 01:46 PM
Last Post: jefsummers
Thumbs Up can't access data from URL in pandas/jupyter notebook aaanoushka 1 2,598 Feb-13-2022, 01:19 PM
Last Post: jefsummers
Question Sorting data with pandas TheZaind 4 3,352 Nov-22-2021, 07:33 PM
Last Post: aserian
  Pandas Data frame column condition check based on length of the value aditi06 1 3,759 Jul-28-2021, 11:08 AM
Last Post: jefsummers
  [Pandas] Write data to Excel with dot decimals manonB 1 8,031 May-05-2021, 05:28 PM
Last Post: ibreeden
  pandas.to_datetime: Combine data from 2 columns ju21878436312 1 3,605 Feb-20-2021, 08:25 PM
Last Post: perfringo
  pandas read_csv can't handle missing data mrdominikku 0 3,584 Jul-09-2020, 12:26 PM
Last Post: mrdominikku
  Pandas data frame creation from Kafka Topic vboppa 0 2,506 Jul-01-2020, 04:23 PM
Last Post: vboppa

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020