Python Forum
Create multiple/single csv file for each sql records
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Create multiple/single csv file for each sql records
#3
Hi Buran,

superb ! Thank you so much , I liked your code. Whistle

Similar situation here, Extracting all records into single csv files.
using fetchmany extracted 60gb of data.
my code is working , can we make any improvement in below code plz

import pyodbc
import csv

connection = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER=DESKTOP-GQK64O6;DATABASE=Customer;Trusted_Connection=yes;')
cursor = connection.cursor()


qry = "select * from employee"

cursor.execute(qry) 
data = cursor.fetchall()


With open("E:\\backup\\output.csv","w","newline="") as outfile
	writer = csv.writer(outfile,quoting = csv.QUOTE_NONNUMERIC)
	writer.writerows(col[0] for col in cursor.description)
	While True:
		rows = cursor.fetchmany(10000)
		if len(rows) ==0:
			print("no records found")
			break
		else:
	          x = x+len(rows)
               print(x)
               for row in rows:
                  writer.writerows(row)


conn.close()
print("success")
print('time taken' . time.time()-initial,"Seconds")
Reply


Messages In This Thread
RE: Create multiple csv file for each sql records - by mg24 - Sep-29-2022, 05:14 AM
Extract sql table data into CSV - by mg24 - Sep-29-2022, 05:22 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Create Choices from .ods file columns cspower 3 732 Dec-28-2023, 09:59 PM
Last Post: deanhystad
  Recommended way to read/create PDF file? Winfried 3 3,043 Nov-26-2023, 07:51 AM
Last Post: Pedroski55
  Use PM4PY and create working file thomaskissas33 0 771 Nov-14-2023, 06:53 AM
Last Post: thomaskissas33
  Create csv file with 4 columns for process mining thomaskissas33 3 863 Nov-06-2023, 09:36 PM
Last Post: deanhystad
  create exe file for linux? korenron 2 1,034 Mar-22-2023, 01:42 PM
Last Post: korenron
  validate large json file with millions of records in batches herobpv 3 1,366 Dec-10-2022, 10:36 PM
Last Post: bowlofred
  my first file won't create itself MehHz2526 2 963 Nov-27-2022, 12:58 AM
Last Post: MehHz2526
  python Multithreading on single file mg24 3 1,872 Nov-05-2022, 01:33 PM
Last Post: snippsat
  Reshaping a single column in to multiple column using Python sahar 7 2,169 Jun-20-2022, 12:35 PM
Last Post: deanhystad
Sad pandas writer create "corrupted" file freko75 1 2,943 Jun-14-2022, 09:57 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020