Python Forum
How Does pyspark deal with Spaces in Queries - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: How Does pyspark deal with Spaces in Queries (/thread-11906.html)

How Does pyspark deal with Spaces in Queries - cpatte7372 - Jul-31-2018

Hello community,

Can someone let me know how pyspark.sql deals with spaces?

For example the following query won't work:

myresults = spark.sql("""SELECT  Name AS [Product Name] FROM Production_vProductAndDescription""")
As you can see above the I have tried to concantenate Product Name with square brackets, but I get error:

Error:
== SQL ==
SELECT  Name AS [Product Name] FROM Production_vProductAndDescription
----------------^^^
However, with no spaces between Product and Name (without the square brackets as follows it works:

myresults = spark.sql("""SELECT  Name AS Product_Name FROM Production_vProductAndDescription""")
Any thoughts welcomed

Cheers

Carlton

RE: How Does pyspark deal with Spaces in Queries - micseydel - Jul-31-2018

I believe this is the same issue as I said in your other thread. It seems like you're used to MSSQL or something and are now using MySQL, Postgres, or something else with a different SQL dialect than you're used to. I don't think this is PySpark-specific.

RE: How Does pyspark deal with Spaces in Queries - cpatte7372 - Jul-31-2018

hi micseydel,

The answer is to change [Product Name] to Product Name

Thanks anyway for reaching out.

Cheers

RE: How Does pyspark deal with Spaces in Queries - micseydel - Jul-31-2018

I understand that. That's consistent with what I'm saying. It's the SQL dialect, not PySpark. I've dabbled with MSSQL, I've never seen the syntax you're trying to use. You need to identify the dialect of the database you're using and familiarize yourself with it, or you're going to hit a great deal of blockers like this.