Python Forum
Python Alteryx QS-Passing pandas dataframe column inside SQL query where condition
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python Alteryx QS-Passing pandas dataframe column inside SQL query where condition
#1
Hi Team
I have a alteryx module that uses python query component

Input to python is 1 column that has ~ 3M values. I read it inside python using Alteryx.read("#1")

This column has 3 M numeric values.

I need to pull all details for these 3 M values from 1 of larger Athena database table 'A' by using where condition on column_A from the table and passing these values as condition on the column_A

I am trying code as below but it is giving me different syntax or logical errors: Please can anyone help me with correcting it or suggesting alternatives to pass these 3 M values of a column into SQL query where condition;

My code which is failing is as follow:

1
2
3
4
5
6
7
8
9
pin_list = Alteryx.read("#1")    (Note: contains 3 Million numeric values like 1 , 2 ,3 etc. )
 
df1= pd.DataFrame(pin_list)
pins= df1['ColumnName'].tolist()
con= pyodbc.connect(DSN= 'Athena' , DATABASE= 'Random')
sql2=  "Select * from A where date = 'randomdate' and column_A in ({}) "\ .format(',  'join(['?' for in pins]))
df= pd.read_sql(sql2, con, params= [pins])
print(df)
Alteryx.write(df,1)
How can i pass 3M numeric values present in 'pins' into where conditon of query on 'column_A' ?
Note: Also Column_A has to be passed With numeric values as required by table 'A' . passing string or characters wont retrieve correct data from the table
buran write Dec-05-2023, 09:27 AM:
Please, use proper tags when post code, traceback, output, etc. This time I have added tags for you.
See BBcode help for more info.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  renaming a column without a name in a dataframe Carbonpony 2 800 Jan-23-2025, 08:20 AM
Last Post: Carbonpony
  Most efficient way to roll through a pandas dataframe? sawtooth500 2 1,078 Aug-28-2024, 10:08 AM
Last Post: Alice12
  docx file to pandas dataframe/excel iitip92 1 2,425 Jun-27-2024, 05:28 AM
Last Post: Pedroski55
  Problems passing arguments containing spaces to bash script and then on to python kaustin 6 8,311 Apr-03-2024, 08:26 PM
Last Post: deanhystad
  Adding PD DataFrame column bsben 2 1,282 Mar-08-2024, 10:46 PM
Last Post: deanhystad
  Get an average of the unique values of a column with group by condition and assign it klllmmm 0 1,901 Feb-17-2024, 05:53 PM
Last Post: klllmmm
  pandas : problem with conditional filling of a column Xigris 2 1,378 Jul-22-2023, 11:44 AM
Last Post: Xigris
  Question on pandas.dataframe merging two colums shomikc 4 1,904 Jun-29-2023, 11:30 AM
Last Post: snippsat
  list the files using query in python arjunaram 0 1,087 Mar-28-2023, 02:39 PM
Last Post: arjunaram
  Pandas AttributeError: 'DataFrame' object has no attribute 'concat' Sameer33 5 10,121 Feb-17-2023, 06:01 PM
Last Post: Sameer33

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020