Python Forum

Full Version: Load data from json into MySQL
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi gurus,

I need to take data from JSON API rest and put it into MySQL database table. I try this with for loop through json list which contains dictionaries for data for transaction but I want it to do width map and lambda function. Loop take some time and this job will run every minutes and I need to take data and load quickly. also I want to put this in asynchronise mode and run the same process for different api rest in the same time. Because that I want to put this logic in map and lambda sytnax, not through for loop syntax.

import mysql.connector as mysql

json_list_of_transactions = [{'amount': '0.05900000', 'date': '1521806276', 'tid': '59790535', 'type': '0', 'price': '8509.96'}, {'amount': '0.04400000', 'date': '1521806270', 'tid': '59790528', 'type': '1', 'price': '8512.08'}, {'amount': '0.03730108', 'date': '1521806268', 'tid': '59790527', 'type': '1', 'price': '8514.21'}]


db = mysql.connect(user='root', password='mysql#1234', host = 'localhost', database='DB_DEV')

cursor = db.cursor()

cursor.execute("TRUNCATE TABLE BTCUSD_TXN_H;")
db.commit()

transaction_sql = (
    "insert into BTCUSD_TXN_H"
    "(DATE_TIMESTAMP, DATE_ID, TID, TYPE, VOLUMEN, PRICE)"
    "values (%(date)s, from_unixtime(%(date)s), %(tid)s, %(type)s, %(amount)s, %(price)s)")

for transaction in json_list_of_transactions:
   cursor.execute(transaction_sql, transaction)

#this for loop I want change and try with lambda and map expression like this 
## (it works without errors but not load data. Why? What is wrong?
#map(lambda transaction: cursor.execute(transaction_sql, transaction), json_list_of_transactions )

db.commit()

cursor.close()
db.close()
(Mar-23-2018, 12:13 PM)chevanton1988 Wrote: [ -> ]
#this for loop I want change and try with lambda and map expression like this 
## (it works without errors but not load data. Why? What is wrong?
#map(lambda transaction: cursor.execute(transaction_sql, transaction), json_list_of_transactions )

map() creates a map-object, which doesn't actually do much of anything until you iterate over it. Check it:
>>> def call_me(param):
...   print("I was called with: {0}".format(param))
...
>>> call_me(42)
I was called with: 42
>>> map(call_me, range(5))
<map object at 0x0374B290>
>>> list(map(call_me, range(5)))
I was called with: 0
I was called with: 1
I was called with: 2
I was called with: 3
I was called with: 4
[None, None, None, None, None]