Python Forum
How to set PYTHONPATH in Visual Studio Code?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to set PYTHONPATH in Visual Studio Code?
#6
(Aug-15-2023, 06:25 AM)aupres Wrote: But it does not works at all. Desperately need your advice.
Do not give path VS Code.
This should work fom commandline before using any Editors.
If i do test install of pyspark using eg conda Installation
conda install -c conda-forge pyspark
I have conda-forge on as default.
# Activate environment  
G:\div_code\egg\ping
λ G:\miniconda3\Scripts\activate.bat tom_env
# Install pyspark
(tom_env) G:\div_code\egg\ping
λ conda install pyspark
Retrieving notices: ...working... done
.....
Test that it work.
(tom_env) G:\div_code\egg\ping
λ ptpython
>>> import pandas as pd
>>> pd.__file__
'G:\\miniconda3\\envs\\tom_env\\lib\\site-packages\\pandas\\__init__.py'

>>> from pyspark.sql import SparkSession
>>> spark = SparkSession.builder.getOrCreate()

# Testing pyspark
>>> from datetime import datetime, date
... import pandas as pd
... from pyspark.sql import Row
...
... df = spark.createDataFrame([
...     Row(a=1, b=2., c='string1', d=date(2000, 1, 1), e=datetime(2000, 1, 1, 12, 0)),
...     Row(a=2, b=3., c='string2', d=date(2000, 2, 1), e=datetime(2000, 1, 2, 12, 0)),
...     Row(a=4, b=5., c='string3', d=date(2000, 3, 1), e=datetime(2000, 1, 3, 12, 0))
... ])
>>> df
DataFrame[a: bigint, b: double, c: string, d: date, e: timestamp]

>>> df.columns
['a', 'b', 'c', 'd', 'e']
So now all work from commandline,without any Editors or setup in them.

For Editors i have to set this,so set PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON.
This is a OS environment setup,and not a setup in Editors.
(tom_env) G:\div_code\egg\ping
λ set PYSPARK_PYTHON=G:\miniconda3\envs\tom_env\python.exe

(tom_env) G:\div_code\egg\ping
λ set PYSPARK_DRIVER_PYTHON=G:\miniconda3\envs\tom_env\python.exe 
Then this work in VS Code with no setup in Editor,other than activate environment.
from datetime import datetime, date
import pandas as pd
from pyspark.sql import Row
from pyspark.sql import SparkSession
spark = SparkSession.builder.getOrCreate()

df = spark.createDataFrame([
    Row(a=1, b=2., c='string1', d=date(2000, 1, 1), e=datetime(2000, 1, 1, 12, 0)),
    Row(a=2, b=3., c='string2', d=date(2000, 2, 1), e=datetime(2000, 1, 2, 12, 0)),
    Row(a=4, b=5., c='string3', d=date(2000, 3, 1), e=datetime(2000, 1, 3, 12, 0))
])

print(df) 

If you messed (base) environment just make a new one,this is important point of using Anaconda.
Do not try to fix a broken environment,just make new one.
Example,this also install new Python 3.10.8 version and notebook i a most have with jupyterlab.
conda create --name my_env -c conda-forge pyspark jupyterlab python=3.10.8 
aupres likes this post
Reply


Messages In This Thread
RE: How to set PYTHONPATH in Visual Studio Code? - by snippsat - Aug-15-2023, 03:51 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  My code works on Jupyter Lab/Notebook, but NOT on Visual Code Editor jst 4 1,112 Nov-15-2023, 06:56 PM
Last Post: jst
  how do I open two instances of visual studio code with the same folder? SuchUmami 3 939 Jun-26-2023, 09:40 AM
Last Post: snippsat
  Visual Studio Code NewPi 3 1,091 May-16-2023, 11:13 PM
Last Post: snippsat
  Visual Studio Code venv ibm_db error mesi1000 7 2,892 Nov-13-2022, 12:36 AM
Last Post: snippsat
  debugging help on Visual Studio Code mg24 12 2,118 Oct-02-2022, 12:18 AM
Last Post: mg24
  Problem with importing Python file in Visual Studio Code DXav 7 5,245 Jun-15-2022, 12:54 PM
Last Post: snippsat
  Visual Studio Code Intellisense for Imported Modules Not Working hockinsk 1 2,766 Apr-23-2022, 04:41 PM
Last Post: deanhystad
Photo Visual studio code unable to color syntax on python interpreter tomtom 4 7,015 Mar-02-2022, 01:23 AM
Last Post: tomtom
  compile error Visual Studio Code jamie_01 2 1,722 Jan-25-2022, 09:36 AM
Last Post: Larz60+
  Calling python from c++ in visual studio pdk5 0 2,191 May-24-2021, 10:18 AM
Last Post: pdk5

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020