Jupyter notebooks, jupyter kernels, virtual environment created in Anaconda...

[bytecrunch]

Hello Forum,

I think I am clear on what a virtual environment is: it is a "folder" that contains specific Python libraries/modules and a specific Python interpreter (we can choose the version).

• Every environment has its own Python interpreter. The purpose of virtual environments is to keep projects and their dependencies separate so there are no conflicts.
  

• Virtual environments, once created, are generally stored all together in a specific and regular folder on our computer. The folder can be anywhere. And the Python scripts that use a specific and activated environment should not be saved inside the environment itself (even if they can be) but in some other folder.
  

• I am using Anaconda and the Anaconda Navigator to create virtual environments at the Anaconda terminal: I start in the (base) virtual environment and then use conda to create a new environment, (ven1):
  
  (base) C:\Users\Tom\python_env\ conda create --name ven1 # to create a new environment called ven1
  (base) C:\Users\Tom\python_env\ conda activate ven1 # to activate the environment ven1
  (ven1) C:\Users\Tom\python_env\ jupyter notebook # this launches jupyter notebook from (ven1). The notebook should use the Python interpreter associated with environment (ven1)? But it doesn't!
  
  

• When I launch jupyter notebook from (ven1), the jupyter kernel for the notebook is the SAME as the kernel that would be used if I launched the Jupyter notebook from the (base) environment. Why? I expected the kernel to be automatically different since (base) and (ven1) have different Python interpreters.
  

• A jupyter kernel is a different name for Python interpreter, correct? So when I launch Jupyter from different environments, shouldn't we there be different kernels since we are using a Jupyter notebook from two different virtual environments having different Python interpreters?
  

• To avoid the problem, we have to run this code to install a new kernel in (ven1)...but doesn't (ven1) already have its own kernel which is different from the kernel of (base)?
  
  
  (ven1) C:\Users\Brett\python_env\ conda install ipykernel # we install the package ipykernel inside the ven1 environment
  (ven1) C:\Users\Brett\python_env\ ipython kernel install --user --name=ven1kernel # creates a new kernel
  (ven1) (C:\Users\Brett\python_env\ jupyter kernelspec list #lists all jupyter kernels
  

In essence, does this all mean that, even if we create a new virtual environment and launch jupyter notebook from it, the kernel that is used, by default, is the kernel (Python interpreter) associated with the (base) environment! This is a problem in the notebook. To avoid that, we need to create a new jupyter kernel and select that kernel in the notebook to be able to use the modules/libraries contained inside the activated environment (ven1). This sounds redundant since (ven1) already has its own Python interpreter but are creating a new one using the library ipykernel

Thank you!

[deanhystad]

A jupyter kernel is a different name for Python interpreter, correct?

No, A jupyter kernel is an interface to the Python interpreter.

After creating your virtual environment did you create a new kernel?

How to do this for jupyter notebook: https://www.geeksforgeeks.org/using-jupy...vironment/

[bytecrunch]

A jupyter kernel is a different name for Python interpreter, correct?

No, A jupyter kernel is an interface to the Python interpreter.

After creating your virtual environment did you create a new kernel?

How to do this for jupyter notebook: https://www.geeksforgeeks.org/using-jupy...vironment/

Well, I did both:

I created a new virtual environment, (ven1), and launched Jupyter, without creating a new jupyter kernel.

Then I created, within that same (ven1) environment, a new jupyter kernel and launched jupyter again after that. This time I had two jupyter kernels available in my notebook...One for the (base) environment and one for (ven1).

Does choosing one kernel versus the other connect different environments to jupyter? Is that what is going on?
Is that the case for other IDEs?

I guess I did not understand that the jupyter kernel is an interface and not an actual Python interpreter and got confused about the fact that (ven1) already has an environment and why jupyter does not use that...

Thank you

[snippsat]

Does choosing one kernel versus the other connect different environments to jupyter? Is that what is going on?

No the are speareted,so have to install eg JupyterLab(which also install the kernel),in each environment.
There had been little point in virtual environment if the shared stuff🧬.
Can do quick demo,here a have older my_env and newer new_env envroment.

C:\Anaconda3
(base) λ activate my_env

C:\Anaconda3
(my_env) λ jupyter --version
Selected Jupyter core packages...
IPython          : 8.9.0
ipykernel        : 6.21.1
ipywidgets       : not installed
jupyter_client   : 7.4.9
jupyter_core     : 5.2.0
jupyter_server   : 2.2.1
jupyterlab       : 3.6.1
nbclient         : 0.7.2
nbconvert        : 7.2.9
nbformat         : 5.7.3
notebook         : 6.5.2
qtconsole        : 5.4.0
traitlets        : 5.9.0

C:\Anaconda3
(my_env) λ activate new_env

conda.bat activate new_env

C:\Anaconda3
(new_env) λ jupyter --version
Selected Jupyter core packages...
IPython          : 8.14.0
ipykernel        : 6.23.1
ipywidgets       : not installed
jupyter_client   : 8.2.0
jupyter_core     : 5.3.0
jupyter_server   : 2.6.0
jupyterlab       : 4.0.1
nbclient         : 0.8.0
nbconvert        : 7.4.0
nbformat         : 5.9.0
notebook         : not installed
qtconsole        : not installed
traitlets        : 5.9.0

# Start JupyterLab from new_env
C:\Anaconda3
(new_env) λ jupyter lab

So in browser can do.

import jupyterlab

jupyterlab.__version__
4.0.1

If i start JupyterLab for my_env it will show version 3.6.1.

For Editors/IDE most choice right environment,eg look at this post for VS Code.
If eg use Spyder you install it to the environment(do not use share one),then start if from command line spyder.

Also a tip try to use command line much to get comfortable with it,many time i just run code from command line in the environment and not involving a editor(of just use to view code).
Then it also help to have something a lot better than cmd/PowerShell like cmder(full)

[bytecrunch]

Does choosing one kernel versus the other connect different environments to jupyter? Is that what is going on?

No the are speareted,so have to install eg JupyterLab(which also install the kernel),in each environment.
There had been little point in virtual environment if the shared stuff🧬.
Can do quick demo,here a have older my_env and newer new_env envroment.

C:\Anaconda3
(base) λ activate my_env

C:\Anaconda3
(my_env) λ jupyter --version
Selected Jupyter core packages...
IPython          : 8.9.0
ipykernel        : 6.21.1
ipywidgets       : not installed
jupyter_client   : 7.4.9
jupyter_core     : 5.2.0
jupyter_server   : 2.2.1
jupyterlab       : 3.6.1
nbclient         : 0.7.2
nbconvert        : 7.2.9
nbformat         : 5.7.3
notebook         : 6.5.2
qtconsole        : 5.4.0
traitlets        : 5.9.0

C:\Anaconda3
(my_env) λ activate new_env

conda.bat activate new_env

C:\Anaconda3
(new_env) λ jupyter --version
Selected Jupyter core packages...
IPython          : 8.14.0
ipykernel        : 6.23.1
ipywidgets       : not installed
jupyter_client   : 8.2.0
jupyter_core     : 5.3.0
jupyter_server   : 2.6.0
jupyterlab       : 4.0.1
nbclient         : 0.8.0
nbconvert        : 7.4.0
nbformat         : 5.9.0
notebook         : not installed
qtconsole        : not installed
traitlets        : 5.9.0

# Start JupyterLab from new_env
C:\Anaconda3
(new_env) λ jupyter lab

So in browser can do.

import jupyterlab

jupyterlab.__version__
4.0.1

If i start JupyterLab for my_env it will show version 3.6.1.

For Editors/IDE most choice right environment,eg look at this post for VS Code.
If eg use Spyder you install it to the environment(do not use share one),then start if from command line spyder.

Also a tip try to use command line much to get comfortable with it,many time i just run code from command line in the environment and not involving a editor(of just use to view code).
Then it also help to have something a lot better than cmd/PowerShell like cmder(full)

Thank you snippsat! I fee like a have done what you describe in your examples. I have used the Anaconda terminal to:

a) create a new environment, (new_env)
b) install jupyter notebook inside of the environment itself
c) launched jupyter notebook from it which then opens up

However, without creating a jupyter kernel specific for that environment via:

(new_env) C:\Users\Brett\python_env\ conda install ipykernel # we install the package ipykernel inside the ven1 environment
(new_env) C:\Users\Brett\python_env\ ipython kernel install --user --name=ven1kernel # creates a new kernel
(new_env) (C:\Users\Brett\python_env\ jupyter kernelspec list #lists all jupyter kernels

then the jupyter notebook will NOT be able to access the modules inside the environment....correct?

[snippsat]

However, without creating a jupyter kernel specific for that environment via:

You don't install a jupyter kernel as mention this is done automatic when install JupyterLab
Here how i like to make a environment,make sure that use channel conda-forge and install a new Python 3.11.4 to this environment.
Make sure all is up to date before making environment conda update --all

conda create --name home_env -c conda-forge spyder jupyterlab pandas python=3.11.4

Now if activate home_env and test:

C:\anaconda3
(base) λ activate home_env

C:\anaconda3
(home_env) λ jupyter --version
Selected Jupyter core packages...
IPython          : 8.15.0
ipykernel        : 6.25.0
ipywidgets       : not installed
jupyter_client   : 7.4.9
jupyter_core     : 5.3.0
jupyter_server   : 1.23.4
jupyterlab       : 3.6.3
nbclient         : 0.5.13
nbconvert        : 6.5.4
nbformat         : 5.9.2
notebook         : 6.5.4
qtconsole        : 5.4.2
traitlets        : 5.7.1

See that ipykernel : 6.25.0 is installed.
If have trouble i would suggest to remove Anaconda and install the new version that are Python 3.11.

[jefsummers]

Maybe I'm not understanding. Using Anaconda Navigator, choose Environments from the menu on the left. Click Create, choose the version of Python you want, give it a name, then add the packages you need for this environment. When you want to launch Jupyter with that environment, simply choose it then go back to apps and pick Jupyter Notebook or Lab. I know you can do it in terminal, but why?

[bytecrunch]

However, without creating a jupyter kernel specific for that environment via:

You don't install a jupyter kernel as mention this is done automatic when install JupyterLab
Here how i like to make a environment,make sure that use channel conda-forge and install a new Python 3.11.4 to this environment.
Make sure all is up to date before making environment conda update --all

conda create --name home_env -c conda-forge spyder jupyterlab pandas python=3.11.4

Now if activate home_env and test:

C:\anaconda3
(base) λ activate home_env

C:\anaconda3
(home_env) λ jupyter --version
Selected Jupyter core packages...
IPython          : 8.15.0
ipykernel        : 6.25.0
ipywidgets       : not installed
jupyter_client   : 7.4.9
jupyter_core     : 5.3.0
jupyter_server   : 1.23.4
jupyterlab       : 3.6.3
nbclient         : 0.5.13
nbconvert        : 6.5.4
nbformat         : 5.9.2
notebook         : 6.5.4
qtconsole        : 5.4.2
traitlets        : 5.7.1

See that ipykernel : 6.25.0 is installed.
If have trouble i would suggest to remove Anaconda and install the new version that are Python 3.11.

Hello snippsat,

I was re-reading your replies. thank you. Let me see if I understood correctly.

ISSUE: my issue was that, after creating a new virtual environment (ven1) and launching jupyter notebook from it (from the CLI of the activated environment), I notices that the notebook did NOT have access to the libraries/modules of (ven1) as I would have liked...

SOLUTIONS
There are two possible solutions:

1) Let's assume we already have Jupyter Notebook installed on our computer. We then create a virtual environment called (ven1). If we launch jupyter notebook from the command line of the activated environment, jupyter notebook will open but the notebook will NOT use the libraries and modules of (ven1), as we would instead like, just because the we opened the app from (ven1).
To solve that, we could install a new jupyter notebook inside (ven1) itself. At that point, when we type "jupyter notebook", the notebook will have access to (ven1) resources...That is redundant: having multiple jupyter notebooks, one for each environment, does not seem efficient.

2) The other and more efficient solution is to:
a) only keep a single Jupyter notebook installation
b) create the new virtual environment (ven1)
c) create a new jupyter kernel inside (ven1) with the steps described above (ipykernel, etc.).
d) after typing "jupyter notebook" at the CLI of activated (ven1), the notebook opens and we simply select the new jupyter kernel specific to (ven1) that we created for the notebook to have access to (ven1) resources...

Did I understand correctly? Thank YOU for your patience!

[snippsat]

It's simple when make a conda environment they are completely stand alone.
So when i make this environment what i have install before dos matter at all.

conda create --name home_env -c conda-forge spyder jupyterlab pandas python=3.11.4

At that point, when we type "jupyter notebook", the notebook will have access to (ven1) resources...That is redundant: having multiple jupyter notebooks, one for each environment, does not seem efficient.

There is no shared Notebook as mention a environment is stand alone as it should be,and you type jupyter lab to start the Notebook.