Error checking and a call to read_cvs

Error checking and a call to read_cvs - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Data Science (https://python-forum.io/forum-44.html)
+--- Thread: Error checking and a call to read_cvs (/thread-33576.html)

Error checking and a call to read_cvs - rsherry8 - May-07-2021

Please consider the following function:

import pandas as pd

def read_input(path1, path2):
    """Concat two paths/strs; yields pandas.DataFrame of indicated file."""
    full_input_path = Path(path1, path2)
    columns = {
        'symbol': str,
        'num_shares': int
    }
    with open(full_input_path) as input_file:
        # Redirect stderr to something we can report on.

            stocks_table = None
            stocks_table = pd.read_csv(
                input_file,
                sep=',',
                names=columns.keys(),
                index_col=False, # fix data file?
                #dtype=columns,
                comment='#',
                error_bad_lines=False, # fix?
                warn_bad_lines=True
                )
         
    # discuss validation
    return stocks_table

The file is suppose to have two fields in it, separated by a comma. However, one of the records is missing
the comma. Hence the call to read_cvs produces the following error message:

Output:
- VO 5: No data found, symbol may be delisted

Is there a way to find out which record in the input was bad?

Bob

RE: Error checking and a call to read_cvs - Larz60+ - May-07-2021

what do path1 and path2 look line before being combined?
what does full_path look like?
Is this what you expected?

Also, when opening a Path type path, do so like (replacement for line 10):

with full_input_path.open() as input_file:

RE: Error checking and a call to read_cvs - rsherry8 - May-08-2021

I tired what you suggested and it did not matter.

The variables path1 and path2 hold the correct values. The file is being opened correctly. The file is missing a comma. The program should and is producing an error message. The problem is that the error message does not contain the line number of the bad line. I want to print an error message telling the user the line number of the bad line. How do I do that?

(May-07-2021, 10:55 PM)Larz60+ Wrote: what do path1 and path2 look line before being combined?
what does full_path look like?
Is this what you expected?

Also, when opening a Path type path, do so like (replacement for line 10):
with full_input_path.open() as input_file:

RE: Error checking and a call to read_cvs - Larz60+ - May-08-2021

Quote:I tired what you suggested and it did not matter.

Please elaborate.

RE: Error checking and a call to read_cvs - rsherry8 - May-08-2021

I updated the code based upon what you suggested. However, I am still not getting the line number of the bad input line in the error message.

(May-08-2021, 04:16 PM)Larz60+ Wrote:
Quote:I tired what you suggested and it did not matter.
Please elaborate.