Python3 + BeautifulSoup4 + lxml (HTML -> CSV) - How to write 3 Columns to MariaDB?

[BrandonKastning]

On line 48, why aren't you extracting the text from allTitle and allOpinion in the same way that you're doing on line 27?

Also, remember to post the full traceback in future, as it contains important info about the error - like the line number it occurs on.

Also avoid using globals and pass the values into your function - the signature was sensible when you originally wrote it; I don't know why you changed that.

ndc85430,

I have since commented line #48 and line #49 reads as the following:

        recordTuple = (allTitle[0], allOpinion[0], allURL)

How do I post a full traceback? Is it a command switch I can append to my normal .py kickoff with my usual "python3 *.py"

Regarding globals; are you referring to "url = allURL" ? Does this qualify as a global variable?

How would I specifically avoid using a global for the use of the function itself?

The current .py looks like the following:

from urllib.request import urlopen
from bs4 import BeautifulSoup
html = urlopen("http://law.justia.com/cases/federal/appellate-courts/F2/999/663/308588/")
bsObj = BeautifulSoup(html.read())
allOpinion = bsObj.findAll(id="opinion")
import requests
from bs4 import BeautifulSoup

url = "http://law.justia.com/cases/federal/appellate-courts/F2/999/663/308588/"
allTitle = bsObj.findAll({"title"})
allURL = url

print(allOpinion)
print(allTitle)
print(allURL)

import csv
csvRow = [allOpinion,allTitle,allURL]
csvfile = "current_F2_opinion_with_tags_current.csv"
with open(csvfile, "a") as fp:
    wr = csv.writer(fp, dialect='excel')
    wr.writerow(csvRow)

print(allOpinion[0].get_text(),url)
 
import csv
csvRow = [allOpinion[0].get_text(),allTitle[0].get_text(),allURL]
csvfile = "current_F2_opinion_without_tags_current.csv"
with open(csvfile, "a") as fp:
    wr = csv.writer(fp, dialect='excel')
    wr.writerow(csvRow)


import mysql.connector
from mysql.connector import Error

#def insertVariblesIntoTable(allTitle, allOpinion, allURL):
def insertVariablesIntoTable():
    try:
        connection = mysql.connector.connect(host='localhost',
                                             database='PythonMariaDB1',
                                             user='PythonMariaDB1',
                                             password='password1234')
        cursor = connection.cursor()
        mySql_insert_query = """INSERT INTO Single_No_Loop (all_Title, all_Opinion, all_URL) 
                                VALUES (%s, %s, %s) """

#        recordTuple = (allTitle, allOpinion, allURL)
        recordTuple = (allTitle[0], allOpinion[0], allURL)
        cursor.execute(mySql_insert_query, recordTuple)
        connection.commit()
        print("Record inserted successfully into Single_No_Loop table")

    except mysql.connector.Error as error:
        print("Failed to insert into MySQL table {}".format(error))

    finally:
        if (connection.is_connected()):
            cursor.close()
            connection.close()
            print("MySQL connection is closed")

insertVariablesIntoTable()

The current error is as follows:

Failed to insert into MySQL table Failed processing format-parameters; Python 'tag' cannot be converted to a MySQL type
MySQL connection is closed