ValueError: Found input variables with inconsistent numbers of samples: [5, 6] - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Data Science (https://python-forum.io/forum-44.html) +--- Thread: ValueError: Found input variables with inconsistent numbers of samples: [5, 6] (/thread-22304.html) |
ValueError: Found input variables with inconsistent numbers of samples: [5, 6] - bongielondy - Nov-07-2019 I am working on a small test data. I am getting a ValueError: Found input variables with inconsistent numbers of samples: [5, 6]. How can I make the X and y shapes to be the same size. I added the line; dataset.dropna(inplace=True) to drop NA values so that the two samples become the same size. However I still get the Value Error. The code is; # Importing Libraries import numpy as np import pandas as pd # Import dataset dataset = pd.read_csv("../output.tsv", delimiter = '\t') # library to clean data import re # Natural Language Tool Kit import nltk nltk.download('stopwords') # to remove stopword from nltk.corpus import stopwords # for Stemming propose from nltk.stem.porter import PorterStemmer # Initialize empty array # to append clean text corpus = [] # 1000 (reviews) rows to clean for i in range(0, 5): # column : "Review", row ith review = re.sub('[^a-zA-Z]', ' ', dataset['Review'][i]) # convert all cases to lower cases review = review.lower() # split to array(default delimiter is " ") review = review.split() # creating PorterStemmer object to # take main stem of each word ps = PorterStemmer() # loop for stemming each word # in string array at ith row review = [ps.stem(word) for word in review if not word in set(stopwords.words('english'))] # rejoin all string array elements # to create back into a string review = ' '.join(review) # append each string to create # array of clean text corpus.append(review) # Creating the Bag of Words model from sklearn.feature_extraction.text import CountVectorizer # To extract max 1500 feature. # "max_features" is attribute to # experiment with to get better results cv = CountVectorizer(max_features = 9) # X contains corpus (dependent variable) X = cv.fit_transform(corpus).toarray() # y contains answers if review # is positive or negative y = dataset.iloc[:, 1].values # Splitting the dataset into # the Training set and Test set from sklearn.model_selection import train_test_split dataset.dropna(inplace=True) print(X.shape) print(y.shape) # experiment with "test_size" # to get better results X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25) print(X_train.shape) print(y_train.shape)The Output from the code (for X shape and y shape) is (5, 9) (6,) Error is ValueError: Found input variables with inconsistent numbers of samples: [5, 6] RE: ValueError: Found input variables with inconsistent numbers of samples: [5, 6] - Larz60+ - Nov-07-2019 Please, always show complete, unmodified error traceback It contains valuable debugging information. RE: ValueError: Found input variables with inconsistent numbers of samples: [5, 6] - bongielondy - Nov-07-2019 The full output is;
RE: ValueError: Found input variables with inconsistent numbers of samples: [5, 6] - kozaizsvemira - Nov-07-2019 If I'm not mistaken your x_train and y_size are different sizes. They have to be equal. Somewhere your output is different than input, working with big data requires all samples to be equal due to the accuracy of tests. RE: ValueError: Found input variables with inconsistent numbers of samples: [5, 6] - bongielondy - Nov-09-2019 The range was incorrect. The file had 6 reviews but the code was; for i in range(0, 5):I have corrected that and code works fine RE: ValueError: Found input variables with inconsistent numbers of samples: [5, 6] - Mahalakshmi - Jun-07-2020 (Nov-07-2019, 03:26 AM)bongielondy Wrote: I am working on a small test data. I am getting a ValueError: Found input variables with inconsistent numbers of samples: [5, 6]. How can I make the X and y shapes to be the same size. I added the line; RE: ValueError: Found input variables with inconsistent numbers of samples: [5, 6] - ricslato - Jun-28-2021 I faced a similar problem while fitting a regression model . The problem in my case was, Number of rows in X was not equal to number of rows in y. You likely get problems because you remove rows containing nulls in X_train and y_train independent of each other. y_train probably has few, or no nulls and X_train probably has some. So when you remove a row in X_train and the same row is not removed in y_train it will cause your data to be unsynced and have different lenghts. Instead you should remove nulls before you separate X and y. |