Python Forum
Strange error ValueError: dimension mismatch
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Strange error ValueError: dimension mismatch
#1
I have to test word2vector model for text data similarity it generate this kind of error ValueError: dimension mismatch

from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
 
tokenizer=Tokenizer()
tokenizer.fit_on_texts(documents_df.documents_cleaned)
tokenized_documents=tokenizer.texts_to_sequences(documents_df.documents_cleaned)
tokenized_paded_documents=pad_sequences(tokenized_documents,maxlen=64,padding='post')
vocab_size=len(tokenizer.word_index)+1
 
# reading Glove word embeddings into a dictionary with "word" as key and values as word vectors
embeddings_index = dict()
 
with open("D:\Clustering\glove.6B.100d.txt", 'r', encoding="utf8") as file:
    for line in file:
        values = line.split()
        word = values[0]
        coefs = np.asarray(values[1:], dtype='float32')
        embeddings_index[word] = coefs
     
# creating embedding matrix, every row is a vector representation from the vocabulary indexed by the tokenizer index. 
embedding_matrix=np.zeros((vocab_size,100))
 
for word,i in tokenizer.word_index.items():
    embedding_vector = embeddings_index.get(word)
    if embedding_vector is not None:
        embedding_matrix[i] = embedding_vector
         
# calculating average of word vectors of a document weighted by tf-idf
document_embeddings=np.zeros((len(tokenized_paded_documents),100))
words=tfidfvectoriser.get_feature_names()
 
# instead of creating document-word embeddings, directly creating document embeddings
for i in range(documents_df.shape[0]):
    for j in range(len(words)):
        document_embeddings[i]+=embedding_matrix[tokenizer.word_index[words[j]]]*tfidf_vectors[i][j]
         
 
pairwise_similarities=cosine_similarity(document_embeddings)
pairwise_differences=euclidean_distances(document_embeddings)
Error:
Error:
Traceback (most recent call last): File "D:/Clustering/text-cluster-master/Cos_Sim_Eucliden_distance.py", line 156, in <module> document_embeddings[i] += embedding_matrix[tokenizer.word_index[words[j]]] * tfidf_vectors[i][j] File "D:\Python3.8.0\Python\lib\site-packages\scipy\sparse\base.py", line 550, in __rmul__ return (self.transpose() * tr).transpose() File "D:\Python3.8.0\Python\lib\site-packages\scipy\sparse\base.py", line 498, in __mul__ raise ValueError('dimension mismatch') ValueError: dimension mismatch
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  ValueError: dimension mismatch Anldra12 0 768 Jul-17-2021, 04:46 PM
Last Post: Anldra12
  ValueError: x and y must have same first dimension, but have shapes (11,) and (15406, hobbyist 17 15,530 Mar-22-2021, 10:27 AM
Last Post: hobbyist
  Why getting ValueError : Math domain error in trig. function, math.asin() ? jahuja73 3 1,145 Feb-24-2021, 05:09 PM
Last Post: bowlofred
  Strange syntax error with f-strings Askic 6 1,288 Oct-16-2020, 10:40 AM
Last Post: Askic
  Error When Plotting ValueError: x and y must have same first dimension JoeDainton123 1 2,757 Oct-04-2020, 12:58 PM
Last Post: scidam
  Receiving ValueError("bad input shape {0}".format(shape)) error SuryaCitizen 2 1,875 Jun-01-2020, 06:45 AM
Last Post: pyzyx3qwerty
  getting error ValueError: time data '' does not match format '%H:%M' srisrinu 2 3,229 Apr-09-2020, 11:12 AM
Last Post: srisrinu
  Exception: Returned Type Mismatch Error devansing 1 2,491 Mar-06-2020, 07:26 PM
Last Post: ndc85430
  Error in the code ->ValueError: could not convert string to float: ' ' eagleboom 1 2,023 Nov-29-2019, 06:19 AM
Last Post: ThomasL
  ValueError: Index contains duplicate entries, cannot reshapeā€¯ error when I try to use Smiling29 11 5,704 Oct-26-2019, 09:52 PM
Last Post: Smiling29

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020