Python Forum
Help with two lines in a function
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Help with two lines in a function
#1
Hi all,

I was wondering if somebody could help me correct two lines in a function i have made to generate a confusion matrix. The lines in question are about reordering columns and indexes as this is required sometimes. In some instances the order of the columns in the confusion matrix need to change. Herein lies the problem: the function defined is as follows: the lines

df_cm = df_cm[[reorderCols]]
df_cm = df_cm.reindex(index=[reorderRows])

is where i get an error when i try and reorder the columns within the function itself.


def kerasConfMat (classLabelPDindex,classLabelPDCols,reorderCols,reorderRows,
                  actual, pred,hmcolor, xlab, ylab, fontsize, titlelab, 
                  xAxislab,yAxislab,titleFontSize, axisLabFontSize, 
                  colorbarlegend, annotatePlot):

    cm = confusion_matrix(actual.argmax(axis=1), pred.argmax(axis=1))
    cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]

    df_cm = pd.DataFrame(
     cm, index=classLabelPDindex, columns=classLabelPDCols)


    df_cm = df_cm[[reorderCols]]
    df_cm = df_cm.reindex(index=[reorderRows])

    ax = plt.axes()

    plot=seaborn.heatmap(df_cm, cmap=hmcolor, xticklabels=xlab, 
                         yticklabels=ylab, ax=ax, 
                         annot=annotatePlot, cbar=colorbarlegend)

    ax.set_title(titlelab, fontsize=titleFontSize)
    plt.xlabel(xAxislab, fontsize=axisLabFontSize)
    plt.ylabel(yAxislab, fontsize=axisLabFontSize)

    plot.yaxis.set_ticklabels(plot.yaxis.get_ticklabels(), rotation=0, 
                              ha='right', fontsize=fontsize)
    plot.xaxis.set_ticklabels(plot.xaxis.get_ticklabels(), rotation=45, 
                              ha='right', fontsize=fontsize)
    return plot

  
  
#Dependencies
import seaborn
from sklearn.metrics import confusion_matrix
in order to re-order the columns and index, I make an order (where the order is currently 3,2,1,5,6,4:
order=["1","2","3","4","5","6"]

and then pass this in to the reorderCols and Rows argument.
When I run this, every argument works fine apart from reorderCols and reorderRows
The error that I receive when having as an argument reorderRows or columns is :

KeyError: "[('1', '2', '3', '4', '5', '6')] not in index"

I can't understand why this isn't working as when I do everything separately (away from a function as follows):

cm = confusion_matrix(y_train[:,0:6].argmax(axis=1), trainpred[:,0:6].argmax(axis=1))

cm = pd.DataFrame(
 cm, index=order, columns=order)

cm=cm[['1', '2', '3', '4', '5', '6']]
cm=cm.reindex(index=['1', '2', '3', '4', '5', '6'])

ax = plt.axes()
cm=seaborn.heatmap(cm, cmap="Greens", xticklabels=ageLabelOrder, yticklabels=ageLabelOrder, annot=True, ax=ax)
this works without a hitch and plots correctly with cols and rowd reordered appropriately. Is there a reason i am getting this error when it is embedded in to a custom function?

any help would be much appreciated!! many thanks!
Reply
#2
check what reorderRows is in your first code example
just by inserting a print(reorderRows) before that line
df_cm = df_cm.reindex(index=[reorderRows])
Looking at your second code example
cm=cm.reindex(index=['1', '2', '3', '4', '5', '6'])
reorderRows must be the same as '1', '2', '3', '4', '5', '6' but if you assign it like this
reorderRows = '1', '2', '3', '4', '5', '6'
reorderRows is of type tuple ('1', '2', '3', '4', '5', '6')
and index=['1', '2', '3', '4', '5', '6'] is not the same as index=[('1', '2', '3', '4', '5', '6')]
which i assume you get printed out using print(reorderRows) as suggested because the error you get
KeyError: "[('1', '2', '3', '4', '5', '6')] not in index"
says exactly this
Reply
#3
(Jul-27-2019, 08:01 PM)ThomasL Wrote: check what reorderRows is in your first code example
just by inserting a print(reorderRows) before that line
df_cm = df_cm.reindex(index=[reorderRows])
Looking at your second code example
cm=cm.reindex(index=['1', '2', '3', '4', '5', '6'])
reorderRows must be the same as '1', '2', '3', '4', '5', '6' but if you assign it like this
reorderRows = '1', '2', '3', '4', '5', '6'
reorderRows is of type tuple ('1', '2', '3', '4', '5', '6')
and index=['1', '2', '3', '4', '5', '6'] is not the same as index=[('1', '2', '3', '4', '5', '6')]
which i assume you get printed out using print(reorderRows) as suggested because the error you get
KeyError: "[('1', '2', '3', '4', '5', '6')] not in index"
says exactly this

Thank you so much!! I am very sorry but i feel like i am doing something so wrong! I get your answer but i continue to get an error.. firstly, print(reorderRows in the first column, does not print the output after execution (snippet of code):

def kerasConfMat (classLabelPDindex,classLabelPDCols,reorderCols,reorderRows,actual, pred,hmcolor, xlab, ylab, fontsize, titlelab, xAxislab,yAxislab,titleFontSize, axisLabFontSize, colorbarlegend, annotatePlot):

    cm = confusion_matrix(actual.argmax(axis=1), pred.argmax(axis=1))
    cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]

    df_cm = pd.DataFrame(
     cm, index=classLabelPDindex, columns=classLabelPDCols)


    df_cm = df_cm[[reorderCols]]
[b]    print(reorderRows)
[/b]    df_cm = df_cm.reindex(index=reorderRows)
After execution I simply get the same error but not output of what reorderRows is:
Traceback (most recent call last):
  File "/Users/jassim01/.virtualenvs/DeannaNN/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3296, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-128-ada69537581b>", line 5, in <module>
    titleFontSize=18, axisLabFontSize=16, colorbarlegend=False, annotatePlot=False)
  File "<ipython-input-126-98aff873c3b0>", line 10, in kerasConfMat
    df_cm = df_cm[[reorderCols]]
  File "/Users/jassim01/.virtualenvs/DeannaNN/lib/python3.7/site-packages/pandas/core/frame.py", line 2682, in __getitem__
    return self._getitem_array(key)
  File "/Users/jassim01/.virtualenvs/DeannaNN/lib/python3.7/site-packages/pandas/core/frame.py", line 2726, in _getitem_array
    indexer = self.loc._convert_to_indexer(key, axis=1)
  File "/Users/jassim01/.virtualenvs/DeannaNN/lib/python3.7/site-packages/pandas/core/indexing.py", line 1327, in _convert_to_indexer
    .format(mask=objarr[mask]))
KeyError: "[('p3', 'p15', 'p30', 'p60', 'p180', 'p365')] not in index"
Secondly, thank you for the clarification that does indeed make perfect sense! I feel like the solution was as follows:

Remove [] from the function so that the argument becomes:

df_cm = df_cm[[reorderCols]]
    print(reorderRows)
    df_cm = df_cm.reindex(index=reorderRows)
then define order=['1','2','3','4','5','6']

whose output =

Out[129]: ['1', '2', '3', '4', '5', '6']

This should make the format of of the function after inserting order in the argument:
df_cm = df_cm.reindex(index=['1', '2', '3', '4', '5', '6'])
Which should be in the correct format, however, i continue to get the same error:

KeyError: "[('p3', 'p15', 'p30', 'p60', 'p180', 'p365')] not in index"

I am really confused as to the specific issue here!
Reply
#4
the error is a line up in df_cm = df_cm[[reorderCols]]
what is reorderCols?
If it is ('p3', 'p15', 'p30', 'p60', 'p180', 'p365') then this is not a proper index
Reply
#5
(Jul-28-2019, 08:50 AM)ThomasL Wrote: the error is a line up in df_cm = df_cm[[reorderCols]] what is reorderCols? If it is ('p3', 'p15', 'p30', 'p60', 'p180', 'p365') then this is not a proper index
Yes!! thank you so much! Even though it was in the callback i thought the actual error was in the index! It works now!

thank you so much for all your assistance :)
Reply
#6
(Jul-28-2019, 09:33 AM)amjass12 Wrote: thank you so much for all your assistance :)
If you appreciate my help then please like my comments and/or give me a reputation.
See below posts.
Thanks. :-)
Reply
#7
(Jul-28-2019, 09:47 AM)ThomasL Wrote:
(Jul-28-2019, 09:33 AM)amjass12 Wrote: thank you so much for all your assistance :)
If you appreciate my help then please like my comments and/or give me a reputation. See below posts. Thanks. :-)
Sure! All done :)
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Importing a function from another file runs the old lines also dedesssse 6 2,478 Jul-06-2021, 07:04 PM
Last Post: deanhystad
  Iterate 2 large text files across lines and replace lines in second file medatib531 13 5,705 Aug-10-2020, 11:01 PM
Last Post: medatib531

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020