Python Forum
Suitable Data Modelling Packages
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Suitable Data Modelling Packages
#1
Hello all

I was wondering if anyone could help me regarding information / guidance on the best packages for data modelling.

I work on the highway and we have lots of faults (cracks, pot holes etc) at various points long different highways.

My idea is to create a data model aims to predict where & when the next fault would occur.

I would feed all existing data such as location, temperature, material, defect size etc for which I have almost 10 years of history of and the model would look for patterns and relationships with the aim of predicting where & when the next fault would occur.

Can I ask what packages would be suitable in order to achieve this, the ones I know I should have are:-

1) Pandas
2) Matplot Lib
3) Numpy

Any help would be greatly appreciated.

Thank you.
Reply
#2
you can look at keras, here's a beginners tutorial: https://victorzhou.com/blog/keras-rnn-tutorial/
Reply
#3
Interesting problem that may be approached in different ways.
For example, if potholes tend to occur in clusters, then a KNN clustering algorithm may be an approach. Scikit-learn has a KNeighborsRegressor to look at as well as other KNN tools.

The same library (scikit-learn) has linear and logistic regression modules for use if you want to predict using that.
Agree with Lars60+ about Keras adding Tensorflow as those have become increasingly wedded in recent releases. The approach with those is a neural network (typically Deep Neural Network or DNN) and that AI approach would be a good alternative to the clustering algorithms and then compare which approach is better.
Reply
#4
algorithms you need depend on the type of data you have and the task to task to achieve;
for example: if you try to predict future events based on past events (this seems your task) you are dealing with a time series forecasting task ;
but the algorithms you choose will depends by: type, quantity and quality of your data;

useful libraries have already been suggested, BUT if you ask this question it means that: you have no idea what to do, and also if we suggest you some useful packages and models for time series, you will have problems in data pre-processing and to know how that algorithms works and how interpret the results ;

usually for TS You can apply autoregressive algorithms like ARIMA, but if you have enough data you can exploit more powerful deep learning models, but especially in this case you must have a firm knowledge of the subject and excellent knowledge of major frameworks like tensorflow and pytorch....
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Stock clustering and portfolio diversification. Suitable features. timurkanaz 1 63 Yesterday, 09:54 AM
Last Post: Larz60+
  predictive modelling beginner1 2 51,902 Sep-27-2019, 09:10 AM
Last Post: beginner1
  Is Python Suitable? summerleas 3 4,037 Feb-28-2017, 12:19 PM
Last Post: merlem

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020