Python Forum
Python Alteryx QS-Passing pandas dataframe column inside SQL query where condition
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python Alteryx QS-Passing pandas dataframe column inside SQL query where condition
#1
Hi Team
I have a alteryx module that uses python query component

Input to python is 1 column that has ~ 3M values. I read it inside python using Alteryx.read("#1")

This column has 3 M numeric values.

I need to pull all details for these 3 M values from 1 of larger Athena database table 'A' by using where condition on column_A from the table and passing these values as condition on the column_A

I am trying code as below but it is giving me different syntax or logical errors: Please can anyone help me with correcting it or suggesting alternatives to pass these 3 M values of a column into SQL query where condition;

My code which is failing is as follow:

pin_list = Alteryx.read("#1")    (Note: contains 3 Million numeric values like 1 , 2 ,3 etc. ) 

df1= pd.DataFrame(pin_list)
pins= df1['ColumnName'].tolist()
con= pyodbc.connect(DSN= 'Athena' , DATABASE= 'Random')
sql2=  "Select * from A where date = 'randomdate' and column_A in ({}) "\ .format(',  'join(['?' for _  in pins])) 
df= pd.read_sql(sql2, con, params= [pins])
print(df)
Alteryx.write(df,1)
How can i pass 3M numeric values present in 'pins' into where conditon of query on 'column_A' ?
Note: Also Column_A has to be passed With numeric values as required by table 'A' . passing string or characters wont retrieve correct data from the table
buran write Dec-05-2023, 09:27 AM:
Please, use proper tags when post code, traceback, output, etc. This time I have added tags for you.
See BBcode help for more info.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Problems passing arguments containing spaces to bash script and then on to python kaustin 6 424 Apr-03-2024, 08:26 PM
Last Post: deanhystad
  Adding PD DataFrame column bsben 2 328 Mar-08-2024, 10:46 PM
Last Post: deanhystad
  Get an average of the unique values of a column with group by condition and assign it klllmmm 0 290 Feb-17-2024, 05:53 PM
Last Post: klllmmm
  pandas : problem with conditional filling of a column Xigris 2 642 Jul-22-2023, 11:44 AM
Last Post: Xigris
  Question on pandas.dataframe merging two colums shomikc 4 848 Jun-29-2023, 11:30 AM
Last Post: snippsat
  list the files using query in python arjunaram 0 682 Mar-28-2023, 02:39 PM
Last Post: arjunaram
  Pandas AttributeError: 'DataFrame' object has no attribute 'concat' Sameer33 5 5,698 Feb-17-2023, 06:01 PM
Last Post: Sameer33
  Difference one column in a dataframe Scott 0 649 Feb-10-2023, 08:41 AM
Last Post: Scott
  splitting a Dataframe Column in two parts nafshar 2 976 Jan-30-2023, 01:19 PM
Last Post: nafshar
  help how to get size of pandas dataframe into MB\GB mg24 1 2,401 Jan-28-2023, 01:23 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020