Python Forum
Insert Pandas Data Frame into Teradata DB
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Insert Pandas Data Frame into Teradata DB
#1
Hi all,

I am using the Teradata python module to read transaction data from Teradata into a Pandas data frame for analysis. Now that I have done my analysis, I need to load the new data frame (rules) back into Teradata. The new data frame does not need to be altered further and only needs to be inserted into Teradata the way it is. I appreciate any help you can provide! Here is my code so far...

#Import Libraries
import teradata
import pandas as pd

#Connect to Teradata 
udaExec = teradata.UdaExec (appName="Teradata_Test", version="1.0",
        logConsole=False)

session = udaExec.connect(method="odbc", system="****",
        username="****", password="****")

#Load the data into a Pandas dataframe  
query = """
   select from where blah blah 
    """

df_transactions_2018 = pd.read_sql(query,session) 

print(df_transactions_2018.head(10))                      
                         
#mlxtend is a machine learning extension package
from mlxtend.preprocessing import TransactionEncoder
from mlxtend.frequent_patterns import apriori
from mlxtend.frequent_patterns import association_rules

#Manipulate the transaction dataframe into a basket dataframe, reset index
df_basket_2018 = df_transactions_2018.groupby('TXN_ID').PROD_ID.apply(list).reset_index()

#Print head df_basket_2018 to check if transaction data is properly manipulated
#1
print(df_basket_2018.head(15))

#Transform Pandas dataframe into a sparse Numpy boolean array with TransactionEncoder to prepare for Apriori algorithm
te = TransactionEncoder()
dset = df_basket_2018.PROD_ID.values
te_ary = te.fit(dset).transform(dset, sparse=True)
StringCols = [str(i) for i in te.columns_]

#Find frequent itemsets through the Apriori algorithm and min_supp. threshold.
aprioriData = pd.SparseDataFrame(te_ary, columns=StringCols, default_fill_value=False)
Results = apriori(aprioriData, min_support=0.01, use_colnames=True, verbose=0)
#2
print(Results)

#Generate association rules from frequent itemsets using a minimum confidence or lift threshold
rules = association_rules(Results, metric="lift", min_threshold=1)

#3
print(rules)

#Load rules dataframe back into Teradata...
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Grouping in pandas/multi-index data frame Aleqsie 3 662 Jan-06-2024, 03:55 PM
Last Post: deanhystad
  Filtering Data Frame, with another value NewBiee 9 1,389 Aug-21-2023, 10:53 AM
Last Post: NewBiee
Smile How to further boost the data read write speed using pandas tjk9501 1 1,263 Nov-14-2022, 01:46 PM
Last Post: jefsummers
  How to insert data in a dataframe? man0s 1 1,326 Apr-26-2022, 11:36 PM
Last Post: jefsummers
Thumbs Up can't access data from URL in pandas/jupyter notebook aaanoushka 1 1,861 Feb-13-2022, 01:19 PM
Last Post: jefsummers
Question Sorting data with pandas TheZaind 4 2,337 Nov-22-2021, 07:33 PM
Last Post: aserian
  Exporting data frame to excel dyerlee91 0 1,626 Oct-05-2021, 11:34 AM
Last Post: dyerlee91
  Pandas Data frame column condition check based on length of the value aditi06 1 2,689 Jul-28-2021, 11:08 AM
Last Post: jefsummers
  Adding a new column to a Panda Data Frame rsherry8 2 2,123 Jun-06-2021, 06:49 PM
Last Post: jefsummers
  [Pandas] Write data to Excel with dot decimals manonB 1 5,869 May-05-2021, 05:28 PM
Last Post: ibreeden

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020