Python Forum

Hi Team,

how to download sql table data with header from using SQL Alchemy.

using pyodbc library I have downloaded 60gb data. Using read in chunk

pandas is slow I think,

from sqlalchemy import create_engine
import pandas as pd
 
#value to connect server
DRIVER= "ODBC Driver 17 for SQL Server"
server="DESKTOP-GQK64O6"
DATABASE="Customer"

#establish connection  
str_conn = f'mssql://{server}/{DATABASE}?Driver={DRIVER}'
connection = create_engine(str_conn).connect()
 
mylist = []
 
#Read in Chunk
for chunk in pd.read_sql_table('employee',connection,chunksize=1000):
    mylist.append(chunk)
df = pd.concat(mylist)
ndf = pd.DataFrame(df)
 
ndf.to_parquet("C:\\C_Programming\\output.csv")

see: https://python-forum.io/thread-24127.html
In this tutorial, I don't write the results to csv file, but that's simple enough.