Keras.Predict into Dataframe - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Data Science (https://python-forum.io/forum-44.html) +--- Thread: Keras.Predict into Dataframe (/thread-24986.html) Pages:
1
2
|
Keras.Predict into Dataframe - Finpyth - Mar-13-2020 Hallo. i'm working with multiple machine learning models. I want to have all my predictions into a dataframe. The following is working for my other ml scripts(sklearn), expect for this one where i use Kera library. It return the error: 'Exception: Data must be 1-dimensional' My DataFrame for all my features is called 'feature_list' and all my targets is 'target_list' import pandas as pd import numpy as np import seaborn as sns import matplotlib.pyplot as plt import statsmodels.api as sm import keras import tensorflow as tf from keras.models import Sequential from keras.layers import Dense from sklearn.metrics import r2_score period = df['2011-01-01':'2015-01-01'] X = np.array(period[feature_list]) predictions = pd.DataFrame() for i in target_list: y = np.array(period[i]) split = sm.add_constant(X) train_size = int(0.75*y.shape[0]) train_features = X[:train_size] train_targets = y[:train_size] test_features = X[train_size:] test_targets = y[train_size:] model = Sequential() model.add(Dense(5,input_dim=train_features.shape[1],activation='relu')) model.add(Dense(5,activation='relu')) model.add(Dense(1,activation='linear')) model.compile(optimizer='adam', loss='mse') model.fit(X, y, epochs=5, verbose=0) print(i) predictions[i] = model.predict(X[train_size:])if i remove '[i]' in 'predictions[i] = model.predict(X[train_size:])' in return some values, so i think the mistake is in the compiling in some way. So how do i compile all my predictions into a dataframe for all my targets? Best regards Fynpyth from Denmark. RE: Keras.Predict into Dataframe - jefsummers - Mar-13-2020 Code is incomplete so cannot test, but I think line 32 should be predictions = model.predict(train_features) and when you are ready to test change to test_features and when you are ready to test change to test_features RE: Keras.Predict into Dataframe - Finpyth - Mar-13-2020 (Mar-13-2020, 05:25 PM)jefsummers Wrote: Code is incomplete so cannot test, but I think line 32 should be Thanks for answer. The "train_features" match the "X[train_size:]" and the code is running for both if i remove '[i]' in the predictions[i], but it's only return one columns of predictions, and i need for all my 40 targets for a trading strategy. I'm sorry i can't upload all my code, but it is for my thesis and my thesis can be flaged as plagiarism if i upload it 1 to 1. sincerly. RE: Keras.Predict into Dataframe - jefsummers - Mar-14-2020 The colon is on the opposite side in train_features vs X[train_size:], so those are not the same thing. After correcting that, do predictions.shape()and post RE: Keras.Predict into Dataframe - Finpyth - Mar-14-2020 (Mar-14-2020, 02:24 AM)jefsummers Wrote: The colon is on the opposite side in train_features vs X[train_size:], so those are not the same thing. You are right about X[train_size:] and train_features is not the same. My bad. I tried your solution but it did not work, but i kinda found a strange 2-step solution. maybe you know why it works? predictions = pd.DataFrame() for i in target_list: y = np.array(period[i]) split = sm.add_constant(X) train_size = int(0.75*y.shape[0]) train_features = X[:train_size] train_targets = y[:train_size] test_features = X[train_size:] test_targets = y[train_size:] model = Sequential() model.add(Dense(50,input_dim=train_features.shape[1],activation='relu')) model.add(Dense(50,activation='relu')) model.add(Dense(1,activation='linear')) model.compile(optimizer='adam', loss='mse') model.fit(X, y, epochs=1, verbose=1) print(i) predictions = pd.DataFrame(model.predict(X[train_size:]), columns=[i])then i remove "predictions = pd.DataFrame()" and add [i] in the last line and get this: for i in target_list: y = np.array(period[i]) split = sm.add_constant(X) train_size = int(0.75*y.shape[0]) train_features = X[:train_size] train_targets = y[:train_size] test_features = X[train_size:] test_targets = y[train_size:] model = Sequential() model.add(Dense(50,input_dim=train_features.shape[1],activation='relu')) model.add(Dense(50,activation='relu')) model.add(Dense(1,activation='linear')) model.compile(optimizer='adam', loss='mse') model.fit(X, y, epochs=1, verbose=1) print(i) predictions[i] = pd.DataFrame(model.predict(X[train_size:]), columns=[i])and now it returns the dataframe, "predictions" with all the predictions for all my targets. If i don't edit the script, it doesn't run best ragards RE: Keras.Predict into Dataframe - jefsummers - Mar-14-2020 The line you remove actually should not do anything - you assign predictions to a blank dataframe and then reassign it in the for loop. Adding the [i] then makes predictions a list of dataframes. I'd be interested if you used type(predictions)after your loop to verify that... RE: Keras.Predict into Dataframe - Finpyth - Mar-14-2020 (Mar-14-2020, 11:45 AM)jefsummers Wrote: The line you remove actually should not do anything - you assign predictions to a blank dataframe and then reassign it in the for loop. Adding the [i] then makes predictions a list of dataframes. I'd be interested if you usedThat was also what I was thinking.type(predictions)after your loop to verify that... It return nothing for the "type(predictions)" In the "variable explorer" in spyder, it say it is a DataFrame. RE: Keras.Predict into Dataframe - jefsummers - Mar-14-2020 May be running into 2 variables - predictions and predictions[]. Idea - go back and in line 1 put predictions[]Then, in your prediction line use predictions.append(pd.DataFrame(model.predict(X[train_size:]), columns=[i]))Then you should have an array of dataframes. You can examine each one, and you can combine them into a single frame later. RE: Keras.Predict into Dataframe - Finpyth - Mar-15-2020 (Mar-14-2020, 04:22 PM)jefsummers Wrote: May be running into 2 variables - predictions and predictions[]. When i run predictions[]in the first line, it return an "invalid syntax". RE: Keras.Predict into Dataframe - jefsummers - Mar-15-2020 predictions = []. Sorry, forgot the equal sign It's just to define predictions as a list so you can use append |