csv Dictwriter problems - Printable Version

csv Dictwriter problems - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: csv Dictwriter problems (/thread-30615.html)

csv Dictwriter problems - fsuedr2022 - Oct-28-2020

I have many lines of data like:

[{'begins_at': '2020-10-22T14:00:00Z',
'open_price': '5.125000',
'close_price': '5.130000',
'high_price': '5.170000',
'low_price': '5.120000',
'volume': 134609,
'session': 'reg',
'interpolated': False,
'symbol': 'PSEC'},
{'begins_at': '2020-10-22T15:00:00Z',
'open_price': '5.135000',
'close_price': '5.140000',
'high_price': '5.150000',
'low_price': '5.130000',
'volume': 48897,
'session': 'reg',
'interpolated': False,
'symbol': 'PSEC'},

When I try to export this to csv using the following:

import csv

with open('twostocks.csv', 'w', newline='') as csvfile:
    fieldnames = [
        'begins_at',
        'open_price',
        'close_price',
        'high_price',
        'low_price',
        'volume',
        'session',
        'interpolated',
        'symbol'
    ]
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames, extrasaction='ignore')
    writer.writeheader()
    writer.writerows(data)

the result yields all of the headers, but the only data output is a single cell containing just 'PSEC' below the 'symbol' column.

<blockquote class="imgur-embed-pub" lang="en" data-id="a/WldrlGx" data-context="false" ><a href="//imgur.com/a/WldrlGx"></a></blockquote><script async src="//s.imgur.com/min/embed.js" charset="utf-8"></script>

RE: csv Dictwriter problems - buran - Oct-28-2020

can you show how you get your data? Your code should work just fine with list of dicts as your sample data. It looks like your data is not what you think - i.e. obviously you scrape them from a website and did not parse correctly. At least not when you try to write them to a file.

RE: csv Dictwriter problems - fsuedr2022 - Oct-28-2020

I am using the following function:

def get_these_historicals(inputSymbols, interval='hour', span='week', bounds='regular', info=None):
    interval_check = ['5minute', '10minute', 'hour', 'day', 'week']
    span_check = ['day', 'week', 'month', '3month', 'year', '5year']
    bounds_check = ['extended', 'regular', 'trading']

    if interval not in interval_check:
        print(
            'ERROR: Interval must be "5minute","10minute","hour","day",or "week"', file=helper.get_output())
        return([None])
    if span not in span_check:
        print('ERROR: Span must be "day","week","month","3month","year",or "5year"', file=helper.get_output())
        return([None])
    if bounds not in bounds_check:
        print('ERROR: Bounds must be "extended","regular",or "trading"', file=helper.get_output())
        return([None])
    if (bounds == 'extended' or bounds == 'trading') and span != 'day':
        print('ERROR: extended and trading bounds can only be used with a span of "day"', file=helper.get_output())
        return([None])

    symbols = helper.inputs_to_set(inputSymbols)
    url = urls.historicals()
    payload = {'symbols': ','.join(symbols),
               'interval': interval,
               'span': span,
               'bounds': bounds}

    data = helper.request_get(url, 'results', payload)
    if (data == None or data == [None]):
        return data

    histData = []
    for count, item in enumerate(data):
        if (len(item['historicals']) == 0):
            print(helper.error_ticker_does_not_exist(symbols[count]), file=helper.get_output())
            continue
        stockSymbol = item['symbol']
        for subitem in item['historicals']:
            subitem['symbol'] = stockSymbol
            histData.append(subitem)

    return(helper.filter(histData, info))

Then I input simply

lookup=('PSEC', 'AAPL')
r.get_these_historicals(lookup)

RE: csv Dictwriter problems - buran - Oct-28-2020

this is not helpful at all as we cannot run it ourselves and reproduce the data. Everything happens within helper not present here. The question is what data and single item looks like
the problem is with the data you get, not the code in the first post. Probably printing data just before you start iterating over it, and printing item should help you debug.

Sorry, it was my mistake. I misunderstood what output you get in the csv file. There is attempt to post some links to imgur and I thought that is what you get in the csv file.
Scratch what I have said so far.

RE: csv Dictwriter problems - buran - Oct-28-2020

with your code

data = [{'begins_at': '2020-10-22T14:00:00Z',
'open_price': '5.125000',
'close_price': '5.130000',
'high_price': '5.170000',
'low_price': '5.120000',
'volume': 134609,
'session': 'reg',
'interpolated': False,
'symbol': 'PSEC'},
{'begins_at': '2020-10-22T15:00:00Z',
'open_price': '5.135000',
'close_price': '5.140000',
'high_price': '5.150000',
'low_price': '5.130000',
'volume': 48897,
'session': 'reg',
'interpolated': False,
'symbol': 'PSEC'}]

import csv

with open('twostocks.csv', 'w', newline='') as csvfile:
    fieldnames = [
        'begins_at',
        'open_price',
        'close_price',
        'high_price',
        'low_price',
        'volume',
        'session',
        'interpolated',
        'symbol'
    ]
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames, extrasaction='ignore')
    writer.writeheader()
    writer.writerows(data)

this is what i get in the csv file

Output:begins_at,open_price,close_price,high_price,low_price,volume,session,interpolated,symbol
2020-10-22T14:00:00Z,5.125000,5.130000,5.170000,5.120000,134609,reg,False,PSEC
2020-10-22T15:00:00Z,5.135000,5.140000,5.150000,5.130000,48897,reg,False,PSEC

so the problem is indeed with the format of your data. check that it is indeed list of dicts as shown