Im trying to save the data scraped to mysql but i have this issues
no more support for MySQLdb, i have to use
1)
on python 3 %s is deprecated and i have to use .format
with the code:
no more support for MySQLdb, i have to use
import pymysql pymysql.install_as_MySQLdb()on settings file for adapt
1)
on python 3 %s is deprecated and i have to use .format
with the code:
def close(self, reason): csv_file = max(glob.iglob('*.csv'), key=os.path.getctime) mydb = MySQLdb.connect(host='localhost', user='demo', passwd='123456', db='testdb') cursor = mydb.cursor() csv_data = csv.reader(open(csv_file)) row_count = 0 for row in csv_data: if row_count != 0: cursor.execute("INSERT IGNORE INTO testtb(product, category) VALUES({},'{}'".format(*row)) row_count += 1 mydb.commit() cursor.close()i have the error
<bound method AutorSpider.close of <AutorSpider 'autor' at 0x7f64725d29b0>> Traceback (most recent call last): File "/home/pc/.local/lib/python3.6/site-packages/twisted/internet/defer.py", line 151, in maybeDeferred result = f(*args, **kw) File "/home/pc/.local/lib/python3.6/site-packages/pydispatch/robustapply.py", line 55, in robustApply return receiver(*arguments, **named) File "/home/pc/Escritorio/fpyautor/fpyautor/spiders/autor.py", line 109, in close cursor.execute("INSERT IGNORE INTO autortb(frase, categoria) VALUES({},'{}')'".format(*row)) File "/home/pc/.local/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute result = self._query(query) File "/home/pc/.local/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query conn.query(q) File "/home/pc/.local/lib/python3.6/site-packages/pymysql/connections.py", line 516, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/home/pc/.local/lib/python3.6/site-packages/pymysql/connections.py", line 727, in _read_query_result result.read() File "/home/pc/.local/lib/python3.6/site-packages/pymysql/connections.py", line 1066, in read first_packet = self.connection._read_packet() File "/home/pc/.local/lib/python3.6/site-packages/pymysql/connections.py", line 683, in _read_packet packet.check_error() File "/home/pc/.local/lib/python3.6/site-packages/pymysql/protocol.py", line 220, in check_error err.raise_mysql_exception(self._data) File "/home/pc/.local/lib/python3.6/site-packages/pymysql/err.py", line 109, in raise_mysql_exception raise errorclass(errno, errval) pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'titulo del item numero 1' at line 1")is other simplier/eficcient way? because im saving the data at the end of the scraping task and if i got more results (3000 items )maybe this is a problem in future with bigger sites?