to_numpy() works in jupyter notebook, but not in python script

to_numpy() works in jupyter notebook, but not in python script - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Data Science (https://python-forum.io/forum-44.html)
+--- Thread: to_numpy() works in jupyter notebook, but not in python script (/thread-19392.html)

to_numpy() works in jupyter notebook, but not in python script - bluefrog - Jun-26-2019

Hi

I am unsure why the following works in a jupyter notebook, but not in a python script that I am run from the Linux command line:

#!/usr/bin/python3
import pandas as pd
import numpy as np

# SWAPPING COLUMNS

dates=pd.date_range('1/1/2019',periods=12)
df=pd.DataFrame(np.random.randn(12,4),index=dates,columns=['A','B','C','D'])

df_copy=df.copy()
# assign, after converting to raw data
df_copy.loc[: ['B', 'A']]=df_copy[['A','B']].to_numpy()
print(df_copy)

Any suggestions ?

RE: to_numpy() works in jupyter notebook, but not in python script - Windspar - Jun-27-2019

List errors .

RE: to_numpy() works in jupyter notebook, but not in python script - scidam - Jun-27-2019

As it could be seen from Pandas official docs .to_numpy was added in v.0.24.0. You are likely using different versions when executing a script and ipynb-file.

RE: to_numpy() works in jupyter notebook, but not in python script - Windspar - Jun-27-2019

to_numpy works fine.

dfn = df[['B', 'A']].to_numpy(copy=True)

This line of code doesn't work for me.

# assign, after converting to raw data
df_copy.loc[: ['B', 'A']]=df_copy[['A','B']].to_numpy()

RE: to_numpy() works in jupyter notebook, but not in python script - scidam - Jun-27-2019

It is very important to post your error messages! I see you are missing a comma in .loc, should be loc[:, ...]. And, what are you trying to do? Switch columns order? You can access underlying numpy array by df.values.

RE: to_numpy() works in jupyter notebook, but not in python script - bluefrog - Jun-27-2019

Thanks for spotting the missing comma.
The error that appears is as follows:

Traceback (most recent call last):
  File "./dataframe_swap_columns.py", line 23, in <module>
    df_copy.loc[:, ['B', 'A']]=df_copy[['A','B']].to_numpy()
  File "/usr/lib/python3/dist-packages/pandas/core/generic.py", line 4378, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'to_numpy'

But as I mentioned, it works in a jupyter notebook.
How can I print the version of pandas in a Python script?
Maybe that is the issue.

RE: to_numpy() works in jupyter notebook, but not in python script - snippsat - Jun-27-2019

import pandas as pd

pd.__version__

As mention by @scidam you need 0.24 --> for .to_numpy() to work.
I guess if you just remove .to_numpy() it will work on earlier version.

Upgrade:

pip3 install --upgrade pandas

Or as me i have pip point to Python 3.7.
Test from command line with -V

tom@tom:~$ python -V
Python 3.7.3

tom@tom:~$ pip -V
pip 19.1 from /home/tom/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip (python 3.7)

RE: to_numpy() works in jupyter notebook, but not in python script - bluefrog - Jun-27-2019

I have 0.23 on my Ubuntu Linux, but 0.24 on my miniconda environment.

I used sudo apt-get install in an attempt to update pandas to 0.24, but I get this message:

$ sudo apt-get install python3-pandas
Reading package lists... Done
Building dependency tree       
Reading state information... Done
python3-pandas is already the newest version (0.23.3+dfsg-3ubuntu1).
The following packages were automatically installed and are no longer required:
..
....
cosmic
Use 'sudo apt autoremove' to remove them.
0 to upgrade, 0 to newly install, 0 to remove and 3 not to upgrade.

So it won't update to 0.24.
At least I know what the issue now.
Thanks for your help.

thanks for the upgrade info.
I had to install pip first using sudo apt-get install python3-pip, and then I was able to
"$ pip3 install --upgrade pandas".
So now the script works.
thanks to everybody for your help.