Python Forum
SQL Alchemy help to extract sql data into csv files
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
SQL Alchemy help to extract sql data into csv files
#1
Hi Team,

how to download sql table data with header from using SQL Alchemy.

using pyodbc library I have downloaded 60gb data. Using read in chunk

pandas is slow I think,



from sqlalchemy import create_engine
import pandas as pd
 
#value to connect server
DRIVER= "ODBC Driver 17 for SQL Server"
server="DESKTOP-GQK64O6"
DATABASE="Customer"

#establish connection  
str_conn = f'mssql://{server}/{DATABASE}?Driver={DRIVER}'
connection = create_engine(str_conn).connect()
 
mylist = []
 
#Read in Chunk
for chunk in pd.read_sql_table('employee',connection,chunksize=1000):
    mylist.append(chunk)
df = pd.concat(mylist)
ndf = pd.DataFrame(df)
 
ndf.to_parquet("C:\\C_Programming\\output.csv")
Reply
#2
see: https://python-forum.io/thread-24127.html
In this tutorial, I don't write the results to csv file, but that's simple enough.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Is it possible to extract 1 or 2 bits of data from MS project files? cubangt 8 1,043 Feb-16-2024, 12:02 AM
Last Post: deanhystad
  Why can't it extract the data from .txt well? Melcu54 3 673 Aug-20-2023, 10:07 PM
Last Post: deanhystad
Question Need help for a python script to extract information from a list of files lephunghien 6 1,076 Jun-12-2023, 05:40 PM
Last Post: snippsat
  script to calculate data in csv-files ledgreve 0 1,092 May-19-2023, 07:24 AM
Last Post: ledgreve
  python Extract sql data by combining below code. mg24 1 958 Oct-03-2022, 10:25 AM
Last Post: mg24
  How to extract specific data from .SRC (note pad file) Shinny_Shin 2 1,263 Jul-27-2022, 12:31 PM
Last Post: Larz60+
  Apply textual data cleaning to several CSV files ErcoleL99 0 832 Jul-09-2022, 03:01 PM
Last Post: ErcoleL99
  Extract parts of multiple log-files and put it in a dataframe hasiro 4 2,082 Apr-27-2022, 12:44 PM
Last Post: hasiro
  Can ZipFile be used to extract hidden files? AiedailEclipsed 0 1,615 Mar-22-2022, 05:21 PM
Last Post: AiedailEclipsed
Question I am struggling with basic json library + sql alchemy w/ mariadb-connector json ->sql BrandonKastning 0 1,510 Mar-04-2022, 09:26 PM
Last Post: BrandonKastning

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020