+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Automating a Data Extraction Process (/thread-2642.html)
Pages:
1
2
Automating a Data Extraction Process - Harrison - Mar-30-2017
Hi everyone,
First of all, this is my first proper post on Python-Forum, so please excuse any poor forum etiquette on my part...
I want to automate a repetitive computational procedure - nothing new, right! Unfortunately, I have no experience in programming, so in fact it is new for me...
Rather than explain everything here, I have put together a PDF which clearly details the steps which I would like to automate using Python:
https://www.dropbox.com/s/ng4573puyyq0o2s/Automating%20Data%20Extraction.pdf?dl=0
Judging from other threads, you guys seem very helpful. This would genuinely be a useful tool for me, which would allow me to easily extract data from the 3D simulation package that I use.
If this program will be too difficult to create in a short time, please let me know why!
Let me know what you think, guys!
Thanks,
Harrison
RE: Automating a Data Extraction Process - Ofnuts - Mar-30-2017
As described, this is just making Python replace your fingers. What are the chances that these applications have a programming interface that could be used directly? Since these apps are professional engineering apps, they likely stick to some standard and there may be python libraries to use the data directly.
RE: Automating a Data Extraction Process - nilamo - Mar-30-2017
As a general rule, we don't write things for people. We help them to figure out how they can do it themselves.
If that's something you're interested in, we can offer tips on how to get started.
If that's NOT what you're after, and was just hoping to have a working solution, there are a couple other directions you can go (such as Fiverr.com).
RE: Automating a Data Extraction Process - Harrison - Mar-30-2017
Hi guys,
Thanks for your replies!
@Ofnuts, I thought the fact that I am constantly jumping between applications would mean that I wouldn't be able to use a specific application's programming interface? Tell me if I'm wrong - I am fully aware that I have no knowledge of such things!
@nilamo, I am happy to learn how to program. I just thought that it was slightly inefficient considering there are probably others who could create an outline of some code for me with relative ease. I appreciate that I'm a newbie and would welcome any help or advice!
I'll pay $80 to someone who can produce a template for me :)
RE: Automating a Data Extraction Process - snippsat - Mar-30-2017
Take a look at Automate the Boring Stuff, Pywinauto.
RE: Automating a Data Extraction Process - Harrison - Mar-30-2017
(Mar-30-2017, 09:06 PM)snippsat Wrote: Take a look at Automate the Boring Stuff, Pywinauto.
Thanks snippsat, will check that out
RE: Automating a Data Extraction Process - Ofnuts - Mar-31-2017
(Mar-30-2017, 08:16 PM)Harrison Wrote: @Ofnuts, I thought the fact that I am constantly jumping between applications would mean that I wouldn't be able to use a specific application's programming interface? Tell me if I'm wrong - I am fully aware that I have no knowledge of such things!The fact that you are jumping across applications only means that you have never considered that there could be ways to do it without a mouse. How it can be done programmatically is however still subject to study and for this one would need to know what each application really does (where are the docs...) and what you are putting in/getting from them. Ideally the whole process can be re-engineered to run without involving the graphical interfaces. Using the window-scraping technique you describe should only be a fall-back plan.
(Mar-30-2017, 08:16 PM)Harrison Wrote: I'll pay $80 to someone who can produce a template for me :)
Reminds me of the proverbial sign in mechanics shops:
Car repairs: $40/hour
Car repairs while you watch: $60/hour
Car repairs while you help: $80/hour
Without seeing your software, if I were a freelancer I would contract for two weeks (and hoping to pull it off in three days of course, but there can be unforeseen problems). Add to this that you have to learn programming, which means 1) learning a language such as Python and 2) acquiring a vast computer culture so that you don't drown under the rain of little problems inherent to this kind of project (file access, text encoding, data representations...).
PS: why photos and not proper screenshots ([prtsc] key)
RE: Automating a Data Extraction Process - Larz60+ - Mar-31-2017
In the 1970's George Morrow owned a company he named Thinker-Toys.
He sold Single Board Computers.
His adds all read similar to this:
Kit $399 Fully Assembled $199
RE: Automating a Data Extraction Process - Harrison - Mar-31-2017
(Mar-31-2017, 12:04 AM)Ofnuts Wrote: The fact that you are jumping across applications only means that you have never considered that there could be ways to do it without a mouse. How it can be done programmatically is however still subject to study and for this one would need to know what each application really does (where are the docs...) and what you are putting in/getting from them. Ideally the whole process can be re-engineered to run without involving the graphical interfaces. Using the window-scraping technique you describe should only be a fall-back plan.
Thanks for the reply, Ofnuts.
Would it be easier to do it programmatically? I assumed that getting a program to click an 'OK' button would be easier than going into the file locations. Each simulation which I am running is saved in a different folder, but in the same drive. The applications automatically detect the specific locations. I appreciate that it would be nice to have the program go to the correct locations but, as I say, I assumed this route may be easier. Again, tell me if I'm wrong!
RE: Automating a Data Extraction Process - Ofnuts - Mar-31-2017
Click an "OK" button by program can be hard. When you are a program you don't really know where some other program's buttons are. If the program uses the standard widgets that can work, but some programs (especially those that have implementations for different OSes) don't.
Several years ago, I read a blog in which the author related that during a mission in some south-american administration he found a guy whose only job was to:
- open a directory in the file explorer
- take a screenshot
- save the screen shot as an image file
- open that file in an OCR program
- save the result (or append it to the existing list if he was scrolling down the directory)
dir *.* >files.txt but the guy made it clear that if he didn't want to lose his job...So, when asking a computer person, come with your raw problem, because the solution you would come up with may not be the most efficient/easy one.