Python Forum
Error checking and a call to read_cvs
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Error checking and a call to read_cvs
#1
Please consider the following function:

import pandas as pd

def read_input(path1, path2):
    """Concat two paths/strs; yields pandas.DataFrame of indicated file."""
    full_input_path = Path(path1, path2)
    columns = {
        'symbol': str,
        'num_shares': int
    }
    with open(full_input_path) as input_file:
        # Redirect stderr to something we can report on.

            stocks_table = None
            stocks_table = pd.read_csv(
                input_file,
                sep=',',
                names=columns.keys(),
                index_col=False, # fix data file?
                #dtype=columns,
                comment='#',
                error_bad_lines=False, # fix?
                warn_bad_lines=True
                )
         
    # discuss validation
    return stocks_table
The file is suppose to have two fields in it, separated by a comma. However, one of the records is missing
the comma. Hence the call to read_cvs produces the following error message:
Output:
- VO 5: No data found, symbol may be delisted
Is there a way to find out which record in the input was bad?

Bob
Reply
#2
what do path1 and path2 look line before being combined?
what does full_path look like?
Is this what you expected?

Also, when opening a Path type path, do so like (replacement for line 10):
with full_input_path.open() as input_file:
Reply
#3
I tired what you suggested and it did not matter.

The variables path1 and path2 hold the correct values. The file is being opened correctly. The file is missing a comma. The program should and is producing an error message. The problem is that the error message does not contain the line number of the bad line. I want to print an error message telling the user the line number of the bad line. How do I do that?

(May-07-2021, 10:55 PM)Larz60+ Wrote: what do path1 and path2 look line before being combined?
what does full_path look like?
Is this what you expected?

Also, when opening a Path type path, do so like (replacement for line 10):
with full_input_path.open() as input_file:
Reply
#4
Quote:I tired what you suggested and it did not matter.
Please elaborate.
Reply
#5
I updated the code based upon what you suggested. However, I am still not getting the line number of the bad input line in the error message.
(May-08-2021, 04:16 PM)Larz60+ Wrote:
Quote:I tired what you suggested and it did not matter.
Please elaborate.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Panda.read_cvs Issues Reading Certain Columns BlackHeart 5 6,089 Oct-27-2017, 04:29 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020