Python Forum
Yahoo_fin, Pandas: how to convert data table structure in csv file
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Yahoo_fin, Pandas: how to convert data table structure in csv file
#11
buran, sorry about that. I will refrain from removing posted content in the future. Thank you for the advice about df.pivot. That was good advice. Also thank you for keeping my posts containing private matters deleted.

BTW, I copied the final code with the 2 lines about csv import and conversion of column to list to the entry above the entry containing the VBA code. I placed the final solution there so that interested people find the final solution quickly.

Regarding the advice to read the csv file into python rather than using Excel VBA to create a string that then needs to be pasted into the python script. That I do not consider an advice. It should be obvious to anybody with just a little programming skills that that is possible and more efficient. As I explained in my now removed post, due to private matters posted, I was just not willing to spend the time figuring out how to do it. I had other priorities. Writing the VBA took me a couple minutes. But eventually my curiosity took over and I did put in the work to learn how to do it in python. Figuring out how to read the csv and that I need to convert the ticker column to a list before it can be further processed took me a couple hours. If I would have been given an example on how this is done conceptually I could have learned it in a few minutes. That would have been advice. Yes, advice should include encouraging to think, but it should much more importantly help in speeding up the learning process. But to your defense I have to add that I discouraged you to give me advice in the form of code. But bottom line, as is, your input should, in my opinion, not counted as advice.

You can delete this reply. You have my consent.
Reply
#12
(Feb-15-2021, 06:05 AM)detlefschmitt Wrote: Regarding the advice to read the csv file into python rather than using Excel VBA to create a string that then needs to be pasted into the python script. That I do not consider an advice. It should be obvious to anybody with just a little programming skills that that is possible and more efficient. As I explained in my now removed post, due to private matters posted, I was just not willing to spend the time figuring out how to do it. I had other priorities. Writing the VBA took me a couple minutes. But eventually my curiosity took over and I did put in the work to learn how to do it in python. Figuring out how to read the csv and that I need to convert the ticker column to a list before it can be further processed took me a couple hours. If I would have been given an example on how this is done conceptually I could have learned it in a few minutes. That would have been advice. Yes, advice should include encouraging to think, but it should much more importantly help in speeding up the learning process. But to your defense I have to add that I discouraged you to give me advice in the form of code. But bottom line, as is, your input should, in my opinion, not counted as advice.

I did show how to read csv file. In my very first post in this thread. So, only you if did pay attention...
(Feb-12-2021, 09:47 AM)buran Wrote:
import pandas as pd
df = pd.read_csv('test.csv')

Then, converting dataframe column to list is google search away. Not that you need to convert to list, you can just iterate over dataframe column values.
By the way, same apply to reading from csv or excel, if you ignored my first post.
So, couple of hours spent on this is your problem, not my fault.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#13
I don't blame you for having to spend a couple hours figuring out a 2-line code.

The problem was not reading the csv. That info is provided as the first line of code on any website explaining how to import a csv. You must be greedy for advice credit if you want to claim that as advice.

The problem was realizing that I need to convert the column to a list. That's what took time. Once I saw that on a website I realized immediately that this is the solution. With some trial and errors I finally came up with the very simple code. If you would have told me that after csv import the ticker column needs to be converted to a list, I would have needed only a fraction of the time. And that would have been a good advice.

Quote:why not learn how to read from a file, e.g. csv/text file with all tickers. instead of constructing a big literal.
And, please, make an effort first. You can read it in number of ways, incl. pandas obviously
That is not an advice.

Quote:I've been working with VBA/Office automation for years. Almost never looked back after I started with python. And you already know how to read data from a file into pandas dataframe and loop over it - what's the problem to read a list of tickers from cav or excel? Basically there is no learning here.
On a broader scale - isn't it better to store data in a DB and then connect from Excel, either with Power Query or VBA/ADODB if you prefer to manipulate your data in Excel?
That is also not an advice.

I am not a bioinformatics person nor an IT guy. At work I never had to deal with DBs that would require the use of Power Query or VBA/ADODB to transfer data to Excel. The only time I used Power Query was to obtain financial statistics data from Yahoo Finance for personal reasons. So, I really don't have much experience with the input and output of data between DBs and Excel. Hence it took time to realize that I need to convert the column to a list. One can have a lot of experience with Excel VBA in one area and not know much in another. On top my knowledge in python is essentially zero.
Reply
#14
So, this is my last post here. This is not about credit for advice (or non-advice). This is about you acting like help vampire from the start and ever so more with each consecutive post.
As I already said, converting to list is not necessary at all in the first place, and second it's a google search away - right from the first result. In any case, it takes less time to do the search, than to write these lengthy posts about getting advice or non-advice. And no where close to spending couple of hours on it.
I don't know how much experience you have with VBA (now it's obvious that not much), so I just asked about why not using alternatives with Power Query or ADODB. You could have ignored it or simply do your research about it, if you are curious.

Anyway, I am done with your threads.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#15
Help vampire. hahahahaha. I think it is exaggerated to call me that after having asked two questions on this forum. I didn't asked for help about importing csv and incorpotaring csv column data into the python script. You volunteered that on your own after I provided the Excel VBA code. I merely said in that post that alternatively instead of using Excel VBA it could be done with python code but I didn't know how to do it. I really didn't want to spend the time to figure it out on my own. So, I left it to somebody else to post it. I did not need the python solution. I was fully happy using the VBA instead.

About my VBA skill. 35 years ago I took a couple semester of Turbo Pascal as an undergraduate. After that I did not program for 25 years. Then 10 years ago we tried in the office to solve the problem of finding key opinion leaders in various disease areas. My opinion was that key opinion leaders can be found in the publication literature and that they could be extracted from the literature database using Excel. I had a very clear idea about how to do that but I had no VBA skills. So, I joined an Excel VBA forum and I broke up the large computation work flow into very small steps. At each step where I didn't know how to code it I asked on the forum. I always asked using an example unrelated to my project that would teach me the concept of a particular programing step. By literally asking a hundred of such questions I finally finished my program. Despite all these questions I absolutely do not think that I acted like a help vampire. The help in the Excel VBA forum was absolutely outstanding and none of the experts complained. By the end of the project I had learned enough to continue developing other programs on my own. I can only recommend learning a programming language this way. It was fun and efficient.

After this first program I developed Excel VBA programs to do text mining, clinical trial data analysis, databasing publication records, various programs to do analysis of chemical and pharmacological data. And most recently I have done a large conceptual work to show that data-to-text (NLG) can be done in Excel using VBA. Data-to-text is a big issue in pharma especially in paperwork preparation for clinical trial data filing. All this was possible because of what I learned in developing the initial key opinion leader program. Yet, I would consider my skill not higher than intermediate. I am aware of various areas in VBA programming where I know very little about. I learn only what I need. After all, programing for me has always remained only a side project at work. I am scientist, not a programmer.

In all my interactions with VBA experts on the Excel VBA forum I never had an awful experience like this one in the python forum. And that basically in my first interaction on this forum. It also never happened on the VBA forum that somebody claimed an advice where there was none. I am the first to give credit where credit is due. But I am also the first to call BS.

That's it for me for this post. No further responses. I suggest you delete all posts below the post with the VBA code.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
Question [SOLVED] Correct way to convert file from cp-1252 to utf-8? Winfried 8 543 Feb-29-2024, 12:30 AM
Last Post: Winfried
  Better python library to create ER Diagram by using pandas data frames as tables klllmmm 0 987 Oct-19-2023, 01:01 PM
Last Post: klllmmm
  Trying to get counts/sum/percentages from pandas similar to pivot table cubangt 6 1,323 Oct-06-2023, 04:32 PM
Last Post: cubangt
Thumbs Up Convert word into pdf and copy table to outlook body in a prescribed format email2kmahe 1 704 Sep-22-2023, 02:33 PM
Last Post: carecavoador
  Using pyodbc&pandas to load a Table data to df tester_V 3 746 Sep-09-2023, 08:55 PM
Last Post: tester_V
  Hard disk structure like a file selection dialog malonn 2 761 Aug-09-2023, 09:14 PM
Last Post: malonn
  Convert File to Data URL michaelnicol 3 1,078 Jul-08-2023, 11:35 AM
Last Post: DeaD_EyE
  Python Script to convert Json to CSV file chvsnarayana 8 2,343 Apr-26-2023, 10:31 PM
Last Post: DeaD_EyE
  python pandas sql table with header mg24 3 1,855 Dec-08-2022, 08:31 PM
Last Post: Larz60+
  openpyxl convert data to float jacklee26 13 5,705 Nov-19-2022, 11:59 AM
Last Post: deanhystad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020