Python Forum
How to use a tfrecord file for training an autoencoder
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to use a tfrecord file for training an autoencoder
#1
I don't know how to get the number of features from a tfrecord file to make them as input to a stacked autoencoder.

I used the follwing function for stacked autoencoder:
from __future__ import print_function
import keras
import numpy
from keras.models import Sequential
from keras.layers.core import *
from sklearn.model_selection import train_test_split
from app_flag import FLAGS

class StackedAutoencoder(object):
    """
    Implementation of stacked autoencoder multi-class classifier using the Keras Python package.
    This classifier is used to classify cells to cell cycle phases S, G1 or G2M.
    """
    def __init__(self, features, labels, num_labels):
        self.features = features
        self.labels = labels
        self.auto_encoder = None
        self.encoding_dim = num_labels
        

        # fix random seed for reproducibility
        self.seed = 7
        numpy.random.seed(7)

    def create_autoencoder(self):
        """
        Build the stacked auto-encoder using multiple hidden layers.
        The stacked auto-encoder is then trained and weights are freezed afterwards.
        A softmax classification layer is that appended to the last layer, replacing the input
        re-constructed layer of the auto-encoder.
        :return: Compiled classification neural network model.
        """
        self.auto_encoder = Sequential()
        self.auto_encoder.add(Dense(3000, activation='relu', input_dim=self.features.shape[1]))
        self.auto_encoder.add(Dense(1000, activation='relu'))
        self.auto_encoder.add(Dense(30, activation='relu'))

        self.auto_encoder.add(Dense(3000, activation='relu'))
        self.auto_encoder.add(Dense(self.features.shape[1], activation='sigmoid'))

        self.auto_encoder.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
        self.auto_encoder.fit(self.features, self.features,
                              epochs=10,
                              batch_size=5,
                              shuffle=True,
                              validation_split=0.33,
                              validation_data=None)

        self.auto_encoder.layers.pop()
        self.auto_encoder.add(Dense(self.encoding_dim, activation='softmax'))
        self.auto_encoder.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
        print(self.auto_encoder.summary())

        # Freeze all weights after training the stacked auto-encoder and all the classification layer
        for i in range(0, len(self.auto_encoder.layers)-1):
            self.auto_encoder.layers[i].trainable = False

        return self.auto_encoder

    def evaluate_autoencoder(self):
        """
        Fit the trained neural network and validate it using splitting the dataset to training and testing sets.
        :return: Accuracy score of the classification.
        """
        self.auto_encoder.fit(self.features, self.labels,
                              epochs=10,
                              batch_size=5,
                              shuffle=True)

        X_train, X_test, Y_train, Y_test = train_test_split(self.features, self.labels, test_size=0.33, random_state=self.seed)
        #predictions = self.auto_encoder.predict_classes(X_test)
        #print(predictions)
        #print(self.label_encoder.inverse_transform(predictions))
        score = self.auto_encoder.evaluate(X_test, Y_test, batch_size=5, verbose=1)
        return score
When I run the code I get an error in the line :

self.auto_encoder.add(Dense(3000, activation='relu', input_dim=self.features.shape[1]))

indicating :TypeError: float() argument must be a string or a number.

So, how can I use the Tfrecord file to get the input dimentionality (the number of features)
Reply


Messages In This Thread
How to use a tfrecord file for training an autoencoder - by JohnMarie - Feb-17-2019, 04:05 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Sample training small model AndrzejB 3 1,248 Mar-22-2023, 07:37 PM
Last Post: jefsummers
  Is it normal so much time training for Training Custom Object Detector?? hobbyist 2 2,810 May-31-2022, 08:55 AM
Last Post: aserikova
  Using Autoencoder for Data Augmentation of numerical Dataset in Python Marvin93 2 3,414 Jul-10-2020, 07:18 PM
Last Post: Marvin93
  How to save predictions made by an autoencoder Glasgow1988 0 1,588 Jul-03-2020, 12:43 PM
Last Post: Glasgow1988
  Differencing Time series and Inverse after Training donnertrud 0 4,143 May-27-2020, 06:11 AM
Last Post: donnertrud
  stacked autoencoder training JohnMarie 0 2,670 Feb-24-2019, 12:23 AM
Last Post: JohnMarie

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020