Python Forum
bulk import in arangodb from a dataframe
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
bulk import in arangodb from a dataframe
#1
Hello

I am trying to bulk import documents to an arangodb collection from dataframes. The code I use is like this:
import pandas as pd
from arango import ArangoClient
import json

client = ArangoClient(hosts='http://127.0.0.1:8529')
db = client.db('DB', username='root', password='123456')

df = pd.read_excel("file.xlsx", sheet_name = "worksheet", header = 1)
df1 = df.to_json(orient="records")
collection = db.collection("collection_name")
collection.import_bulk(df1)
When I run this code I get the following error:
Error:
DocumentInsertError: [HTTP 400][ERR 1227] invalid document type
Reading the documentation I saw this:
Quote:DataFrame.to_json
Parameters:
path_or_bufstr or file handle, optional
File path or object. If not specified, the result is returned as a string.
If I run the above code like this:
df1 = df.to_json("filename.json",orient="records")
with open("filename.json", 'r') as json_file:
    data = json.load(json_file)
collection = db.collection("collection_name")
collection.import_bulk(data)

then the data are imported in the database without any problems. Can Anyone help and explain why this is happening? I am trying to import a huge amount of data in the database and writing the json files takes too much time and sometimes creates other problems.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Bulk Insert into SQL Server without a file rsouza 0 2,450 Oct-04-2018, 02:02 PM
Last Post: rsouza
  Trying to import JSON data into Python/Pandas DataFrame then edit then write CSV Rhubear 0 4,098 Jul-23-2018, 09:50 PM
Last Post: Rhubear

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020