Python Forum
Python Alteryx QS-Passing pandas dataframe column inside SQL query where condition
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python Alteryx QS-Passing pandas dataframe column inside SQL query where condition
#1
Hi Team
I have a alteryx module that uses python query component

Input to python is 1 column that has ~ 3M values. I read it inside python using Alteryx.read("#1")

This column has 3 M numeric values.

I need to pull all details for these 3 M values from 1 of larger Athena database table 'A' by using where condition on column_A from the table and passing these values as condition on the column_A

I am trying code as below but it is giving me different syntax or logical errors: Please can anyone help me with correcting it or suggesting alternatives to pass these 3 M values of a column into SQL query where condition;

My code which is failing is as follow:

pin_list = Alteryx.read("#1")    (Note: contains 3 Million numeric values like 1 , 2 ,3 etc. ) 

df1= pd.DataFrame(pin_list)
pins= df1['ColumnName'].tolist()
con= pyodbc.connect(DSN= 'Athena' , DATABASE= 'Random')
sql2=  "Select * from A where date = 'randomdate' and column_A in ({}) "\ .format(',  'join(['?' for _  in pins])) 
df= pd.read_sql(sql2, con, params= [pins])
print(df)
Alteryx.write(df,1)
How can i pass 3M numeric values present in 'pins' into where conditon of query on 'column_A' ?
Note: Also Column_A has to be passed With numeric values as required by table 'A' . passing string or characters wont retrieve correct data from the table
buran write Dec-05-2023, 09:27 AM:
Please, use proper tags when post code, traceback, output, etc. This time I have added tags for you.
See BBcode help for more info.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  renaming a column without a name in a dataframe Carbonpony 2 752 Jan-23-2025, 08:20 AM
Last Post: Carbonpony
  Most efficient way to roll through a pandas dataframe? sawtooth500 2 1,064 Aug-28-2024, 10:08 AM
Last Post: Alice12
  docx file to pandas dataframe/excel iitip92 1 2,377 Jun-27-2024, 05:28 AM
Last Post: Pedroski55
  Problems passing arguments containing spaces to bash script and then on to python kaustin 6 8,242 Apr-03-2024, 08:26 PM
Last Post: deanhystad
  Adding PD DataFrame column bsben 2 1,275 Mar-08-2024, 10:46 PM
Last Post: deanhystad
  Get an average of the unique values of a column with group by condition and assign it klllmmm 0 1,884 Feb-17-2024, 05:53 PM
Last Post: klllmmm
  pandas : problem with conditional filling of a column Xigris 2 1,375 Jul-22-2023, 11:44 AM
Last Post: Xigris
  Question on pandas.dataframe merging two colums shomikc 4 1,895 Jun-29-2023, 11:30 AM
Last Post: snippsat
  list the files using query in python arjunaram 0 1,083 Mar-28-2023, 02:39 PM
Last Post: arjunaram
  Pandas AttributeError: 'DataFrame' object has no attribute 'concat' Sameer33 5 10,065 Feb-17-2023, 06:01 PM
Last Post: Sameer33

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020