I have the following code to retrieve a sentiment from the given sentence using the FinBert model:
MAX_LEN = 160
class_names = ['negative', 'neutral', 'positive']
encoded_new = tokenizer.encode_plus(
review_text, # Sentence to encode.
add_special_tokens = True, # Add '[CLS]' and '[SEP]'
max_length = MAX_LEN, # Pad & truncate all sentences.
pad_to_max_length = True,
return_attention_mask = True, # Construct attn. masks.
return_tensors = 'pt', # Return pytorch tensors.
)
# Add the encoded sentence to the list.
input_idst = (encoded_new['input_ids'])
attention_maskst = (encoded_new['attention_mask'])
# Convert the lists into tensors.
input_idst = torch.cat([input_idst], dim=0)
attention_maskst = torch.cat([attention_maskst], dim=0)
new_test_output = model(input_idst, token_type_ids=None,
attention_mask=attention_maskst)
logits = new_test_output[0]
predicted = logits.detach().numpy()
# Store predictions
flat_predictions = np.concatenate(predicted, axis=0)
# For each sample, pick the label (0 or 1) with the higher score.
new_predictions = np.argmax(flat_predictions).flatten()
class_names[new_predictions[0]]
How can I apply this code to a whole column in a csv dataset (for each row) instead of an individual sentence and get predictions?
Sorry for the trivial question, since I am relatively new to Python. Thank you!
MAX_LEN = 160
class_names = ['negative', 'neutral', 'positive']
encoded_new = tokenizer.encode_plus(
review_text, # Sentence to encode.
add_special_tokens = True, # Add '[CLS]' and '[SEP]'
max_length = MAX_LEN, # Pad & truncate all sentences.
pad_to_max_length = True,
return_attention_mask = True, # Construct attn. masks.
return_tensors = 'pt', # Return pytorch tensors.
)
# Add the encoded sentence to the list.
input_idst = (encoded_new['input_ids'])
attention_maskst = (encoded_new['attention_mask'])
# Convert the lists into tensors.
input_idst = torch.cat([input_idst], dim=0)
attention_maskst = torch.cat([attention_maskst], dim=0)
new_test_output = model(input_idst, token_type_ids=None,
attention_mask=attention_maskst)
logits = new_test_output[0]
predicted = logits.detach().numpy()
# Store predictions
flat_predictions = np.concatenate(predicted, axis=0)
# For each sample, pick the label (0 or 1) with the higher score.
new_predictions = np.argmax(flat_predictions).flatten()
class_names[new_predictions[0]]
How can I apply this code to a whole column in a csv dataset (for each row) instead of an individual sentence and get predictions?
Sorry for the trivial question, since I am relatively new to Python. Thank you!