Im trying to implement TF_IDF for a task in Python and i just found this cod in net for computing IDF but it doesnt seem to work.
thanks for the edits. sorry it was my first time posting here.
def compute_idf(self , documents): import math idfdic = {} N = len(documents) idfDict = dict.fromkeys(documents[0].keys(), 0) for doc in documents: for word,val in doc.items(): if val>0: idfdic[word]+=1 for word, val in idfDict.items(): idfDict[word] = math.log10(N / float(val)) return idfDict
Error:AttributeError: 'int' object has no attribute 'items'
can anyone please help me with it?thanks for the edits. sorry it was my first time posting here.
Error:6 print ("Computing features...")
7
----> 8 feature_comp = FeatureComputer(train)
9 data_train = feature_comp.get_features_train()
10 data_test = feature_comp.get_features_test(test)
<ipython-input-38-e321c3668b7d> in __init__(self, documents)
3 self.docs = self.load_documents(documents)
4 self.vocab = self.extract_vocabulary()
----> 5 self.idf = self.compute_idf(self.docs)
6 self.vocab_index = self.get_vocab_index()
7
<ipython-input-38-e321c3668b7d> in compute_idf(self, documents)
50 idfDict = dict.fromkeys(documents[0].keys(), 0)
51 for doc in documents:
---> 52 for word,val in doc.items():
53 if val>0:
54 idfdic[word]+=1