How to change from printFacts ( ) to return a list & Loop over list when writing CSV

[Ivan1]

So based on this i would assume if you make a list and append fact.effective_numeric_value or fact.normalized_value (based on whatever you want) then you would get all the values instead of the last one. Then return the list instead of factValue.

Thank you, that sounds very logical, but as I'm completely new to Python, I would not know how I can make those changes. I will try though. I assume the code that I need to modify is below?

def factFinder( instance, namespace, label ):
	# Locate facts in the instance document by namespace and label, ignoring facts that have a context with a segment_element
	l = []
	for f in instance.items:
		if f.qname.namespace_name.find( namespace ) and f.qname.local_name == label:
			l.append( f )
	if not l:
		print('### NOT FOUND', label, type(l))
	return l
	
	
def printFacts( facts, indent=1, targetDate=None ):
    # Find the fact for the relevant target date and print it
    factValue = 0
    for fact in facts:
        if targetDate==None or fact.context.period.instant == targetDate:
            if fact.concept.item_type==fact.concept.MONETARY_ITEM_TYPE:
                factValue = fact.effective_numeric_value
                print( indent * "\t", camelToSpaces( fact.qname.local_name ).ljust(100-indent*8), "$", '{0:>16,}'.format( factValue ) )
            else:
                factValue = fact.normalized_value
                print( indent * "\t", camelToSpaces( fact.qname.local_name ).ljust(100-indent*8), factValue )
    return factValue

[metulburr]

I cant really verify that this works since i dont have "altova"

def printFacts( facts, indent=1, targetDate=None ):
    my_list = []
    for fact in facts:
        if targetDate==None or fact.context.period.instant == targetDate:
            if fact.concept.item_type==fact.concept.MONETARY_ITEM_TYPE:
                my_list.append(fact.effective_numeric_value)
                #print( indent * "\t", camelToSpaces( fact.qname.local_name ).ljust(100-indent*8), "$", '{0:>16,}'.format( factValue ) )
            else:
                my_list.append(fact.normalized_value)
                #print( indent * "\t", camelToSpaces( fact.qname.local_name ).ljust(100-indent*8), factValue )
    return my_list

If you want the prints uncommented you would have to remove this {0:>16,} to just {} as thats not going to work on a list like it did a single string value

[Ivan1]

Thank you very much. I updated the script with your code. I did receive the below error (which I think you were expecting to see as indicated in your earlier post)

Error:

Warning: An error occured while processing Python script 'file:///Users/ivanstamenkovic/Desktop/SEC/extractRatios.py'!
Reason: TypeError: can only concatenate list (not "str") to list
Details:
python-traceback: File "/Users/ivanstamenkovic/Desktop/SEC/extractRatios.py", line 149, in on_xbrl_valid
ratiofile.write( tradingSymbol + "," + entityRegistrantName + "," + entityCentralIndexKey + "," + currentFiscalYearEndDate + "," + docType + "," + documentPeriodEndDate + "," + documentFiscalYearFocus + "," + documentFiscalPeriodFocus + "," + entityCommonStockSharesOutstanding + "," + documentCreationDate + "," + commonStockSharesIssued + "," + treasuryStockShares + "," + commonStockSharesOutstanding + "\n")
python-error: An error occured while processing Python script 'file:///Users/ivanstamenkovic/Desktop/SEC/extractRatios.py'!

Line 149 in script is:

ratiofile.write( tradingSymbol + "," + entityRegistrantName + "," + entityCentralIndexKey + "," + currentFiscalYearEndDate + "," + docType + "," + documentPeriodEndDate + "," + documentFiscalYearFocus + "," + documentFiscalPeriodFocus + "," + entityCommonStockSharesOutstanding + "," + documentCreationDate + "," + commonStockSharesIssued + "," + treasuryStockShares + "," + commonStockSharesOutstanding + "\n")

Do you have any suggestions? I will also read the Python standard module (CSV writing 14.1) in case I can find how to fix.

[Larz60+]

which version of python are you using?
In python 3
I would split line 149, and use the format statement:

with open("ratios.csv", "a") as ratiofile:
    buff = '{},{},{},{},{},{},{},{},{},{},{},{},{}\n'.format(tradingSymbol, entityRegistrantName, entityCentralIndexKey,
                                                          currentFiscalYearEndDate,
                                                          docType, documentPeriodEndDate, documentFiscalYearFocus,
                                                          documentFiscalPeriodFocus,
                                                          entityCommonStockSharesOutstanding, documentCreationDate,
                                                          commonStockSharesIssued, treasuryStockShares,
                                                          commonStockSharesOutstanding)
    ratiofile.write(buff)

if python 3.6 (and newer)

        with open("ratios.csv", "a") as ratiofile:
with open("ratios.csv", "a") as ratiofile:
    buff = f'{tradingSymbol},{entityRegistrantName},{entityCentralIndexKey},{currentFiscalYearEndDate},' \
           f'{docType},{documentPeriodEndDate},{documentFiscalYearFocus},{documentFiscalPeriodFocus},' \
           f'{entityCommonStockSharesOutstanding},{documentCreationDate},{commonStockSharesIssued},' \
           f'{treasuryStockShares},{commonStockSharesOutstanding}\n'
    ratiofile.write(buff)

You don't need the close statement when using with, file is closed automatically.

(not tested- may need minor tweeking)

[metulburr]

ratiofile.write( tradingSymbol + "," + entityRegistrantName + "," + entityCentralIndexKey + "," + currentFiscalYearEndDate + "," + docType + "," + documentPeriodEndDate + "," + documentFiscalYearFocus + "," + documentFiscalPeriodFocus + "," + entityCommonStockSharesOutstanding + "," + documentCreationDate + "," + commonStockSharesIssued + "," + treasuryStockShares + "," + commonStockSharesOutstanding + "\n")

I would also write this as Larz suggested (most likely the first one since the latter is newer). The author looks like they came from C/C++ and not much python.

Anyways, this line is all concatenating strings and values that are strings. So now that it returns a list, one (or many) of those now might be a list. I can see at least one of them is as one is the return value of the function printFacts *docType"

        docType = printFacts( documentType )
        entityName = printFacts( entityRegistrantName )
        entityCIK = printFacts( entityCentralIndexKey )

doctype would now be a list, not a string anymore as printFacts returns lists now


Its going to mess up the layout of the csv file. Im not sure what way you want it. The following would convert the list values to a string joined by a comma , via the first line in the code below. For every value that is now a list would have to be done such as this. Now sure how many as they are all on the same line.

docType = ','.join(docType)
with open("ratios.csv", "a") as ratiofile:
    buff = '{},{},{},{},{},{},{},{},{},{},{},{},{}\n'.format(tradingSymbol, entityRegistrantName, entityCentralIndexKey,
                                                          currentFiscalYearEndDate,
                                                          docType, documentPeriodEndDate, documentFiscalYearFocus,
                                                          documentFiscalPeriodFocus,
                                                          entityCommonStockSharesOutstanding, documentCreationDate,
                                                          commonStockSharesIssued, treasuryStockShares,
                                                          commonStockSharesOutstanding)
    ratiofile.write(buff)