Sep-30-2022, 12:14 PM
Hi Team,
how to download sql table data with header from using SQL Alchemy.
using pyodbc library I have downloaded 60gb data. Using read in chunk
pandas is slow I think,
how to download sql table data with header from using SQL Alchemy.
using pyodbc library I have downloaded 60gb data. Using read in chunk
pandas is slow I think,
from sqlalchemy import create_engine import pandas as pd #value to connect server DRIVER= "ODBC Driver 17 for SQL Server" server="DESKTOP-GQK64O6" DATABASE="Customer" #establish connection str_conn = f'mssql://{server}/{DATABASE}?Driver={DRIVER}' connection = create_engine(str_conn).connect() mylist = [] #Read in Chunk for chunk in pd.read_sql_table('employee',connection,chunksize=1000): mylist.append(chunk) df = pd.concat(mylist) ndf = pd.DataFrame(df) ndf.to_parquet("C:\\C_Programming\\output.csv")