Python Forum

So ive been working on a scrapying project and up until now everything has been working great and as expected. Now that we have successfully scrapped the website we wanted, we wanted to port it over to a database to store the data instead writing to a csv file.

My attempt has failed due to a cursor error within my scrapy pipelines.py file.

here is my code: (Below the code is the error im getting)

    def _init_(self):
        try:
            self.mariadb_connection = mariadb.connect(user='test', password='test', database='test')
            self.cursor = self.mariadb_connection.cursor()
            
        except mariadb.Error as e:
            raise e
    
#    def close_spider(self, spider):
#        ## clean up when spider is closed
#        self.client.close()
        
    def process_item(self, item, spider):
        try:
            [b]self.cursor.execute[/b]( "INSERT INTO test (style,brand,description,price,compurl,reviewcount,reviewrating,model,packsize,spechandle,specbladelength,specoveralllength,specweight) VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)", 
            (item['style'],
            item['brand'],                    
            item['description'],
            item['price'],
            item['compurl'],
            item['reviewcount'],
            item['reviewrating'],
            item['model'],
            item['spechandle'],
            item['specbladelength'],
            item['specoveralllength'],
            item['specweight'],
            item['packsize'])) 
            self.conn.commit()
            
        except mariadb.Error as e:
            print("Error %d: %s" % (e.args[0], e.args[1]))
            return item

AttributeError: 'DsgDbPipeline' object has no attribute 'cursor'

---

I went ahead and moved the connection lines around and got it to work..
Basically move the connection and cursor down into the process_item and that worked..

The reason DsgDbPipeline object has no attribute cursor is because the init did not happen because

def _init_(self):

should have double underscores

def __init__(self):

so should i re-enable the init properly formatted and run it that way from now on?

Yes, if you fix the init it will work as originally intended.