California Public Salary Files - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: California Public Salary Files (/thread-2586.html) |
California Public Salary Files - Larz60+ - Mar-26-2017 California posts files containing salary and compensation of many (all?) public positions on one of their websites. These files are all of the same .csv format which makes them very handy for various applications. I have been playing with them, and thanks to Snippsat am able to scrape the ASP.net catalog. Keeping the list up to date was a pain, so I created a GUI to aid in the process. In the next few days, I will add:
Here's a screenshot: [attachment=144] RE: California Public Salary Files - Larz60+ - Mar-27-2017 Added automatic unpacking of zip files RE: California Public Salary Files - snippsat - Mar-27-2017 You should add a way to run it. Eg: pip freeze > requirements.txt This contain: appdirs==1.4.3 beautifulsoup4==4.5.3 lxml==3.7.3 packaging==16.8 pyparsing==2.2.0 requests==2.13.0 six==1.10.0Then run instruction would be: git clone https://github.com/Larz60p/CaliforniaPublicSalaries.git pip install -r requirements.txt CaliforniaPublicSalaries\src\>python CaCompGui.pyThis is one way of doing it,other more normal way is pip install my_module .Then setup.py do it all,find and install 3-party requirements.
RE: California Public Salary Files - Larz60+ - Mar-27-2017 As soon as I wake up completely from my nap (I never used to do that), I am going to follow your advice. It's high time I learned (and put into practice) how to create packages. It's been on my list for quite some time. |