How to add multiple tables to pyspark sql

[cpatte7372]

Hello community,

Can someone let me know how to add multiple tables to a my query?

As you can see from the code below I have two tables i) Person_Person ii) appl_stock. The problem is the code won't work with the two tables. It will only work with single table. I have tried the following but it didn't work.

df = spark.read.csv('/home/packt/Downloads/Spark_DataFrames/"Person_Person.csv", "appl_stock.csv"',inferSchema=True,header=True)

#%%
import findspark
findspark.init('/home/packt/spark-2.1.0-bin-hadoop2.7')
import pyspark
from pyspark.sql import SparkSession
spark = SparkSession.builder.appName('ops').getOrCreate()
df = spark.read.csv('/home/packt/Downloads/Spark_DataFrames/Person_Person.csv, appl_stock.csv',inferSchema=True,header=True)
df.createOrReplaceTempView('Person_Person, appl_stock')
results = spark.sql("SELECT \
appl_stock.Open\
, appl_stock.Close\
 FROM appl_stock\
 WHERE appl_stock.Close < 500")
carl = spark.sql("SELECT * FROM Person_Person")
results.show()

Any help will be greatly appreciated.

Cheers

Carlton

[Larz60+]

Normal SQL would be to use an Inner Join.
You can find a spark example here: http://bailiwick.io/2015/07/12/joining-d...spark-sql/

[cpatte7372]

Larz60+

Thanks for reaching out. I will check out the link you provided..

In the meantime, I'm happy for this question to be closed.

Cheers