May-03-2019, 03:19 PM
I've accidentally created an infinite loop importing a CSV file using python and importing it into MySQL. I don't see where the error is!
This is my code:
It's a set of unique numbers that identify AWS accounts.
I'm verifying that the data is being inserted by logging into the database and watching it with this query:
Why is this code looping infinitely???
This is my code:
import os import mysql.connector import csv import pandas source = os.path.join('source_files', 'aws_bills', 'march-bill-original-2019.csv') destination = os.path.join('output_files', 'aws_bills', 'aws-bill-2019-03.csv') mydb = mysql.connector.connect(user='xxxx', password='xxxxx', host='xxxxx', database='aws_bill') cursor = mydb.cursor() try: with open(source) as csv_file: csv_reader = csv.reader(csv_file, delimiter=',') next(csv_reader) sql = "INSERT INTO billing_info (InvoiceId, PayerAccountId, LinkedAccountId, RecordType, RecordId, ProductName, RateId, SubscriptionId, PricingPlanId, UsageType, Operation, AvailabilityZone, ReservedInstance, ItemDescription, UsageStartDate, UsageEndDate, UsageQuantity, BlendedRate, BlendedCost, UnBlendedRate, UnBlendedCost, ResourceId, Engagement, Name, Owner, Parent) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)" for row in csv_reader: print(row) try: cursor.execute(sql.format(row)) except Exception as e: print("MySQL Exception: ", e) mydb.commit() print("Done importing data.") except Exception as e: print("Exception:", e) mydb.rollback() finally: mydb.close()The main field that I'm using to identify what is being inserted into the database is the one called LinkedAccountId.
It's a set of unique numbers that identify AWS accounts.
I'm verifying that the data is being inserted by logging into the database and watching it with this query:
select ID,LinkedAccountId from billing_info ORDER BY ID DESC LIMIT 25;After a while of importing, I see that it starts again from the top of the CSV file. The LinkedAccountIds that were reported before start appearing again. Indicating that it has started over from the beginning of the file.
Why is this code looping infinitely???