Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
numpy dtype anomaly
#1
I'm attempting to load 2 arrays from 2 columns read from a file . The file is delimited and I'm using numpy's loadtxt() function to load the arrays, like so:

#!/usr/bin/python3

import sys
import numpy as np
import os.path as op
from datetime import datetime, date, time
from io import StringIO

sample_data = StringIO("AAPL,28-01-2011, ,344.17,344.4,333.53,336.1,21144800\n\
AAPL,31-01-2011, ,335.8,340.04,334.3,339.32,13473000\n\
AAPL,01-02-2011, ,341.3,345.65,340.98,345.03,15236800\n\
AAPL,02-02-2011, ,344.45,345.25,343.55,344.32,9242600\n\
AAPL,03-02-2011, ,343.8,344.24,338.55,343.44,14064100\n\
AAPL,04-02-2011, ,343.61,346.7,343.51,346.5,11494200")

def usage():
    print("usage: {} {}".format(op.basename(sys.argv[0], 'filename')))

def get_weekday(date_str):
    return datetime.strptime(date_str, "%d-%m-%Y").date().weekday()

def load_arrays(data_file, *col_tuple):
    a1 = a2 = None
    rec_type = np.dtype([('stock_code', '|S4'), ('cob_date', '|S10'), ('filler', '|S1'), 
                        ('low_price', 'f4'), ('high_price', 'f4'), ('close_price', 'f4'), 
                        ('valuation', 'f4'), ('volume', 'uint') ])

    try:
        a1, a2 = np.loadtxt(data_file, dtype=rec_type, usecols=col_tuple, delimiter=',', unpack=True)
        # a1, a2 = np.loadtxt(data_file, usecols=col_tuple, delimiter=',', unpack=True)
    except IOError as e:
        usage() # failed to open file
    except Exception as e: print(e)

    return a1, a2

try:
    # data_file = sys.argv[0]
    data_file = sample_data
    c, v = load_arrays(data_file, 5, 6)
except IndexError:
    usage()

print("Closing price array:\n{}".format(c))
print("\nValuation array:\n{}".format(v))
When I attempt to load the arrays without any data types defined then the load is successfull,
i.e. using
a1, a2 = np.loadtxt(data_file, usecols=col_tuple, delimiter=',', unpack=True)
but when I attempt to apply data types, by specifying
a1, a2 = np.loadtxt(data_file, dtype=rec_type, usecols=col_tuple, delimiter=',', unpack=True)
I get the following output
list index out of range
Closing price array:
None

Valuation array:
None
Can anybody suggest why the difference or what I am specifying incorrectly as part of the data type specification?
Reply


Messages In This Thread
numpy dtype anomaly - by bluefrog - Nov-05-2018, 09:09 PM
RE: numpy dtype anomaly - by stullis - Nov-06-2018, 12:29 AM
RE: numpy dtype anomaly - by bluefrog - Nov-07-2018, 12:00 AM
RE: numpy dtype anomaly - by stullis - Nov-07-2018, 12:14 AM
RE: numpy dtype anomaly - by bluefrog - Nov-07-2018, 12:37 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  [Numpy] How to store different data type in one numpy array? water 7 706 Mar-26-2024, 02:18 PM
Last Post: snippsat
  FutureWarning: Logical ops (and, or, xor) between Pandas objects and dtype-less seque NewBiee 5 1,687 Sep-12-2023, 03:15 PM
Last Post: deanhystad
  Numpy returns "TypeError: unsupported operand type(s) for *: 'numpy.ufunc' and 'int'" kalle 2 2,675 Jul-19-2022, 06:31 AM
Last Post: paul18fr
  Data dtype error according to the rule "safe" AndreasMavro 5 9,252 Feb-27-2020, 10:46 PM
Last Post: Pama
  dtype in not working in mad() function ift38375 8 4,048 Jul-22-2019, 02:53 AM
Last Post: scidam
  "erlarge" a numpy-matrix to numpy-array PhysChem 2 3,030 Apr-09-2019, 04:54 PM
Last Post: PhysChem
  ValueError: Input contains infinity or a value too large for dtype('float64') Rabah_r 1 12,946 Apr-06-2019, 11:08 AM
Last Post: scidam

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020