Python Forum
beautiful soup - parsing scraped code in a script
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
beautiful soup - parsing scraped code in a script
#1
Hi, I'm currently working on a project that needs the price of an item at a certain date. using the bs4 module, I have parsed this code:

<script>
                function scaleLabelFilter(number) {
                    var decPlaces = Math.pow(10,1), abbrev = ["k", "m", "b", "t"];
                    for (var i=abbrev.length-1; i>=0; i--) {
                        var size = Math.pow(10,(i+1)*3);
                        if(size <= number) {
                            number = Math.round(number*decPlaces/size)/decPlaces;
                            if((number == 1000) && (i < abbrev.length - 1)) {
                                number = 1;
                                i++;
                            }
                            number += abbrev[i];
                            break;
                        }
                    }

                    return number;
                }
                                var average30 = [['Date','Daily','Average']], average90 = [['Date','Daily','Average']], average180 = [['Date','Daily','Average']], trade30 =[['Date','Total']], trade90 =[['Date','Total']], trade180 =[['Date','Total']];
                                        average30.push([new Date('2018/02/08'), 3, 3]);
                                        trade30.push([new Date('2018/02/08'), 3169]);
                                        average30.push([new Date('2018/02/09'), 3, 3]);
                                        trade30.push([new Date('2018/02/09'), 12407]);
                                        average30.push([new Date('2018/02/10'), 3, 3]);
                                        trade30.push([new Date('2018/02/10'), 2601]);
                                        average30.push([new Date('2018/02/11'), 4, 3]);
                                        trade30.push([new Date('2018/02/11'), 7582]);
                                        average30.push([new Date('2018/02/12'), 4, 3]);
                                        trade30.push([new Date('2018/02/12'), 506]);
                                        average30.push([new Date('2018/02/13'), 5, 3]);
                                        trade30.push([new Date('2018/02/13'), 4643]);
                                        average30.push([new Date('2018/02/14'), 5, 3]);
                                        trade30.push([new Date('2018/02/14'), 102]);
                                        average30.push([new Date('2018/02/15'), 5, 3]);
                                        trade30.push([new Date('2018/02/15'), 10427]);
                                        average30.push([new Date('2018/02/16'), 4, 3]);
                                        trade30.push([new Date('2018/02/16'), 7405]);
                                        average30.push([new Date('2018/02/17'), 4, 3]);
                                        trade30.push([new Date('2018/02/17'), 7757]);
                                        average30.push([new Date('2018/02/18'), 4, 3]);
                                        trade30.push([new Date('2018/02/18'), 7511]);
                                        average30.push([new Date('2018/02/19'), 4, 3]);
                                        trade30.push([new Date('2018/02/19'), 9603]);
                                        average30.push([new Date('2018/02/20'), 4, 3]);
                                        trade30.push([new Date('2018/02/20'), 3932]);
                                        average30.push([new Date('2018/02/21'), 5, 3]);
                                        trade30.push([new Date('2018/02/21'), 1418]);
                                        average30.push([new Date('2018/02/22'), 5, 3]);
                                        trade30.push([new Date('2018/02/22'), 8728]);
                                        average30.push([new Date('2018/02/23'), 5, 3]);
                                        trade30.push([new Date('2018/02/23'), 7731]);
                                        average30.push([new Date('2018/02/24'), 5, 3]);
                                        trade30.push([new Date('2018/02/24'), 2986]);
                                        average30.push([new Date('2018/02/25'), 5, 3]);
                                        trade30.push([new Date('2018/02/25'), 7450]);
                                        average30.push([new Date('2018/02/26'), 5, 3]);
                                        trade30.push([new Date('2018/02/26'), 7999]);
                                        average30.push([new Date('2018/02/27'), 4, 3]);
                                        trade30.push([new Date('2018/02/27'), 14039]);
                                        average30.push([new Date('2018/02/28'), 3, 3]);
                                        trade30.push([new Date('2018/02/28'), 5352]);
                                        average30.push([new Date('2018/03/01'), 4, 3]);
                                        trade30.push([new Date('2018/03/01'), 3022]);
                                        average30.push([new Date('2018/03/02'), 5, 3]);
                                        trade30.push([new Date('2018/03/02'), 11631]);
                                        average30.push([new Date('2018/03/03'), 6, 3]);
                                        trade30.push([new Date('2018/03/03'), 12587]);
                                        average30.push([new Date('2018/03/04'), 6, 4]);
                                        trade30.push([new Date('2018/03/04'), 2936]);
                                        average30.push([new Date('2018/03/05'), 7, 4]);
                                        trade30.push([new Date('2018/03/05'), 7598]);
                                        average30.push([new Date('2018/03/06'), 8, 4]);
                                        trade30.push([new Date('2018/03/06'), 11157]);
                                        average30.push([new Date('2018/03/07'), 8, 4]);
                                        trade30.push([new Date('2018/03/07'), 6153]);
                                        average30.push([new Date('2018/03/08'), 9, 4]);
                                        trade30.push([new Date('2018/03/08'), 1021]);
                        average90.push([new Date('2017/12/10'), 3, 2]);
                                        trade90.push([new Date('2017/12/10'), 1010]);
                        average90.push([new Date('2017/12/11'), 3, 2]);
                                        trade90.push([new Date('2017/12/11'), 9703]);
                        average90.push([new Date('2017/12/12'), 2, 2]);
                                        trade90.push([new Date('2017/12/12'), 7709]);
                        average90.push([new Date('2017/12/13'), 2, 2]);
                                        trade90.push([new Date('2017/12/13'), 5017]);
                        average90.push([new Date('2017/12/14'), 3, 2]);
                                        trade90.push([new Date('2017/12/14'), 3664]);
                        average90.push([new Date('2017/12/15'), 3, 2]);
                                        trade90.push([new Date('2017/12/15'), 8062]);
                        average90.push([new Date('2017/12/16'), 3, 2]);
                                        trade90.push([new Date('2017/12/16'), 6200]);
                        average90.push([new Date('2017/12/17'), 3, 2]);
                                        trade90.push([new Date('2017/12/17'), 3795]);
                        average90.push([new Date('2017/12/18'), 4, 2]);
                                        trade90.push([new Date('2017/12/18'), 6141]);
                        average90.push([new Date('2017/12/19'), 5, 2]);
                                        trade90.push([new Date('2017/12/19'), 3670]);
                        average90.push([new Date('2017/12/20'), 5, 3]);
                                        trade90.push([new Date('2017/12/20'), 5359]);
                        average90.push([new Date('2017/12/21'), 5, 3]);
                                        trade90.push([new Date('2017/12/21'), 8826]);
                        average90.push([new Date('2017/12/22'), 5, 3]);
                                        trade90.push([new Date('2017/12/22'), 7256]);
                        average90.push([new Date('2017/12/23'), 5, 3]);
                                        trade90.push([new Date('2017/12/23'), 4241]);
                        average90.push([new Date('2017/12/24'), 6, 3]);
                                        trade90.push([new Date('2017/12/24'), 3912]);
                        average90.push([new Date('2017/12/25'), 6, 3]);
                                        trade90.push([new Date('2017/12/25'), 2923]);
                        average90.push([new Date('2017/12/26'), 6, 3]);
                                        trade90.push([new Date('2017/12/26'), 10002]);
                        average90.push([new Date('2017/12/27'), 5, 3]);
                                        trade90.push([new Date('2017/12/27'), 7760]);
                        average90.push([new Date('2017/12/28'), 5, 3]);
                                        trade90.push([new Date('2017/12/28'), 4760]);
                        average90.push([new Date('2017/12/29'), 5, 3]);
                                        trade90.push([new Date('2017/12/29'), 8245]);
                        average90.push([new Date('2017/12/30'), 5, 3]);
                                        trade90.push([new Date('2017/12/30'), 16014]);
                        average90.push([new Date('2017/12/31'), 5, 3]);
                                        trade90.push([new Date('2017/12/31'), 4475]);
                        average90.push([new Date('2018/01/01'), 5, 3]);
                                        trade90.push([new Date('2018/01/01'), 6562]);
                        average90.push([new Date('2018/01/02'), 5, 3]);
                                        trade90.push([new Date('2018/01/02'), 5610]);
                        average90.push([new Date('2018/01/03'), 5, 4]);
                                        trade90.push([new Date('2018/01/03'), 7359]);
                        average90.push([new Date('2018/01/04'), 6, 4]);
                                        trade90.push([new Date('2018/01/04'), 5222]);
                        average90.push([new Date('2018/01/05'), 6, 4]);
                                        trade90.push([new Date('2018/01/05'), 7991]);
                        average90.push([new Date('2018/01/06'), 6, 4]);
                                        trade90.push([new Date('2018/01/06'), 9680]);
                        average90.push([new Date('2018/01/07'), 6, 4]);
                                        trade90.push([new Date('2018/01/07'), 6059]);
                        average90.push([new Date('2018/01/08'), 6, 4]);
                                        trade90.push([new Date('2018/01/08'), 6093]);
                        average90.push([new Date('2018/01/09'), 6, 4]);
                                        trade90.push([new Date('2018/01/09'), 14197]);
                        average90.push([new Date('2018/01/10'), 5, 4]);
                                        trade90.push([new Date('2018/01/10'), 6972]);
                        average90.push([new Date('2018/01/11'), 4, 4]);
                                        trade90.push([new Date('2018/01/11'), 8424]);
                        average90.push([new Date('2018/01/12'), 4, 4]);
                                        trade90.push([new Date('2018/01/12'), 8803]);
                        average90.push([new Date('2018/01/13'), 4, 4]);
                                        trade90.push([new Date('2018/01/13'), 13273]);
                        average90.push([new Date('2018/01/14'), 4, 5]);
                                        trade90.push([new Date('2018/01/14'), 7609]);
                        average90.push([new Date('2018/01/15'), 4, 5]);
                                        trade90.push([new Date('2018/01/15'), 6227]);
                        average90.push([new Date('2018/01/16'), 4, 5]);
                                        trade90.push([new Date('2018/01/16'), 18825]);
                        average90.push([new Date('2018/01/17'), 3, 5]);
                                        trade90.push([new Date('2018/01/17'), 1439]);
                        average90.push([new Date('2018/01/18'), 3, 4]);
                                        trade90.push([new Date('2018/01/18'), 9854]);
                        average90.push([new Date('2018/01/19'), 3, 4]);
                                        trade90.push([new Date('2018/01/19'), 3525]);
                        average90.push([new Date('2018/01/20'), 3, 4]);
                                        trade90.push([new Date('2018/01/20'), 7759]);
                        average90.push([new Date('2018/01/21'), 3, 4]);
                                        trade90.push([new Date('2018/01/21'), 4282]);
                        average90.push([new Date('2018/01/22'), 4, 4]);
                                        trade90.push([new Date('2018/01/22'), 7970]);
                        average90.push([new Date('2018/01/23'), 3, 4]);
                                        trade90.push([new Date('2018/01/23'), 5571]);
                        average90.push([new Date('2018/01/24'), 3, 4]);
                                        trade90.push([new Date('2018/01/24'), 1825]);
                        average90.push([new Date('2018/01/25'), 3, 4]);
                                        trade90.push([new Date('2018/01/25'), 2877]);
                        average90.push([new Date('2018/01/26'), 4, 4]);
                                        trade90.push([new Date('2018/01/26'), 5258]);
                        average90.push([new Date('2018/01/27'), 4, 4]);
                                        trade90.push([new Date('2018/01/27'), 4483]);
                        average90.push([new Date('2018/01/28'), 4, 4]);
                                        trade90.push([new Date('2018/01/28'), 982]);
                        average90.push([new Date('2018/01/29'), 4, 4]);
                                        trade90.push([new Date('2018/01/29'), 9984]);
                        average90.push([new Date('2018/01/30'), 4, 4]);
                                        trade90.push([new Date('2018/01/30'), 1003]);
                        average90.push([new Date('2018/01/31'), 4, 4]);
                                        trade90.push([new Date('2018/01/31'), 8589]);
                        average90.push([new Date('2018/02/01'), 4, 4]);
                                        trade90.push([new Date('2018/02/01'), 11749]);
                        average90.push([new Date('2018/02/02'), 3, 4]);
                                        trade90.push([new Date('2018/02/02'), 17634]);
                        average90.push([new Date('2018/02/03'), 2, 4]);
                                        trade90.push([new Date('2018/02/03'), 25650]);
                        average90.push([new Date('2018/02/04'), 2, 3]);
                                        trade90.push([new Date('2018/02/04'), 1165]);
                        average90.push([new Date('2018/02/05'), 3, 3]);
                                        trade90.push([new Date('2018/02/05'), 5894]);
                        average90.push([new Date('2018/02/06'), 3, 3]);
                                        trade90.push([new Date('2018/02/06'), 17497]);
                        average90.push([new Date('2018/02/07'), 2, 3]);
                                        trade90.push([new Date('2018/02/07'), 7445]);
                        average90.push([new Date('2018/02/08'), 3, 3]);
                                        trade90.push([new Date('2018/02/08'), 3169]);
                        average90.push([new Date('2018/02/09'), 3, 3]);
                                        trade90.push([new Date('2018/02/09'), 12407]);
                        average90.push([new Date('2018/02/10'), 3, 3]);
                                        trade90.push([new Date('2018/02/10'), 2601]);
                        average90.push([new Date('2018/02/11'), 4, 3]);
                                        trade90.push([new Date('2018/02/11'), 7582]);
                        average90.push([new Date('2018/02/12'), 4, 3]);
                                        trade90.push([new Date('2018/02/12'), 506]);
                        average90.push([new Date('2018/02/13'), 5, 3]);
                                        trade90.push([new Date('2018/02/13'), 4643]);
                        average90.push([new Date('2018/02/14'), 5, 3]);
                                        trade90.push([new Date('2018/02/14'), 102]);
                        average90.push([new Date('2018/02/15'), 5, 3]);
                                        trade90.push([new Date('2018/02/15'), 10427]);
                        average90.push([new Date('2018/02/16'), 4, 3]);
                                        trade90.push([new Date('2018/02/16'), 7405]);
                        average90.push([new Date('2018/02/17'), 4, 3]);
                                        trade90.push([new Date('2018/02/17'), 7757]);
                        average90.push([new Date('2018/02/18'), 4, 3]);
                                        trade90.push([new Date('2018/02/18'), 7511]);
                        average90.push([new Date('2018/02/19'), 4, 3]);
                                        trade90.push([new Date('2018/02/19'), 9603]);
                        average90.push([new Date('2018/02/20'), 4, 3]);
                                        trade90.push([new Date('2018/02/20'), 3932]);
                        average90.push([new Date('2018/02/21'), 5, 3]);
                                        trade90.push([new Date('2018/02/21'), 1418]);
                        average90.push([new Date('2018/02/22'), 5, 3]);
                                        trade90.push([new Date('2018/02/22'), 8728]);
                        average90.push([new Date('2018/02/23'), 5, 3]);
                                        trade90.push([new Date('2018/02/23'), 7731]);
                        average90.push([new Date('2018/02/24'), 5, 3]);
                                        trade90.push([new Date('2018/02/24'), 2986]);
                        average90.push([new Date('2018/02/25'), 5, 3]);
                                        trade90.push([new Date('2018/02/25'), 7450]);
                        average90.push([new Date('2018/02/26'), 5, 3]);
                                        trade90.push([new Date('2018/02/26'), 7999]);
                        average90.push([new Date('2018/02/27'), 4, 3]);
                                        trade90.push([new Date('2018/02/27'), 14039]);
                        average90.push([new Date('2018/02/28'), 3, 3]);
                                        trade90.push([new Date('2018/02/28'), 5352]);
                        average90.push([new Date('2018/03/01'), 4, 3]);
                                        trade90.push([new Date('2018/03/01'), 3022]);
                        average90.push([new Date('2018/03/02'), 5, 3]);
                                        trade90.push([new Date('2018/03/02'), 11631]);
                        average90.push([new Date('2018/03/03'), 6, 3]);
                                        trade90.push([new Date('2018/03/03'), 12587]);
                        average90.push([new Date('2018/03/04'), 6, 4]);
                                        trade90.push([new Date('2018/03/04'), 2936]);
                        average90.push([new Date('2018/03/05'), 7, 4]);
                                        trade90.push([new Date('2018/03/05'), 7598]);
                        average90.push([new Date('2018/03/06'), 8, 4]);
                                        trade90.push([new Date('2018/03/06'), 11157]);
                        average90.push([new Date('2018/03/07'), 8, 4]);
                                        trade90.push([new Date('2018/03/07'), 6153]);
                        average90.push([new Date('2018/03/08'), 9, 4]);
                                        trade90.push([new Date('2018/03/08'), 1021]);
                        average180.push([new Date('2017/09/11'), 3, 3]);
                                        trade180.push([new Date('2017/09/11'), 4895]);
                        average180.push([new Date('2017/09/12'), 3, 3]);
                                        trade180.push([new Date('2017/09/12'), 4360]);
                        average180.push([new Date('2017/09/13'), 3, 3]);
                                        trade180.push([new Date('2017/09/13'), 26]);
                        average180.push([new Date('2017/09/14'), 3, 3]);
                                        trade180.push([new Date('2017/09/14'), 7841]);
                        average180.push([new Date('2017/09/15'), 3, 3]);
                                        trade180.push([new Date('2017/09/15'), 2559]);
                        average180.push([new Date('2017/09/16'), 3, 3]);
                                        trade180.push([new Date('2017/09/16'), 6881]);
                        average180.push([new Date('2017/09/17'), 3, 3]);
                                        trade180.push([new Date('2017/09/17'), 10]);
                        average180.push([new Date('2017/09/18'), 3, 3]);
                                        trade180.push([new Date('2017/09/18'), 6722]);
                        average180.push([new Date('2017/09/19'), 2, 3]);
                                        trade180.push([new Date('2017/09/19'), 4931]);
                        average180.push([new Date('2017/09/20'), 2, 3]);
                                        trade180.push([new Date('2017/09/20'), 669]);
                        average180.push([new Date('2017/09/21'), 2, 3]);
                                        trade180.push([new Date('2017/09/21'), 4611]);
                        average180.push([new Date('2017/09/22'), 2, 3]);
                                        trade180.push([new Date('2017/09/22'), 5691]);
                        average180.push([new Date('2017/09/23'), 2, 3]);
                                        trade180.push([new Date('2017/09/23'), 4091]);
                        average180.push([new Date('2017/09/24'), 2, 3]);
                                        trade180.push([new Date('2017/09/24'), 0]);
                        average180.push([new Date('2017/09/25'), 2, 3]);
                                        trade180.push([new Date('2017/09/25'), 3806]);
                        average180.push([new Date('2017/09/26'), 2, 3]);
                                        trade180.push([new Date('2017/09/26'), 4701]);
                        average180.push([new Date('2017/09/27'), 2, 2]);
                                        trade180.push([new Date('2017/09/27'), 3562]);
                        average180.push([new Date('2017/09/28'), 2, 2]);
                                        trade180.push([new Date('2017/09/28'), 9176]);
                        average180.push([new Date('2017/09/29'), 2, 2]);
                                        trade180.push([new Date('2017/09/29'), 7318]);
                        average180.push([new Date('2017/09/30'), 3, 2]);
                                        trade180.push([new Date('2017/09/30'), 3262]);
                        average180.push([new Date('2017/10/01'), 4, 2]);
                                        trade180.push([new Date('2017/10/01'), 1547]);
                        average180.push([new Date('2017/10/02'), 4, 2]);
                                        trade180.push([new Date('2017/10/02'), 8709]);
                        average180.push([new Date('2017/10/03'), 3, 2]);
                                        trade180.push([new Date('2017/10/03'), 5301]);
                        average180.push([new Date('2017/10/04'), 3, 2]);
                                        trade180.push([new Date('2017/10/04'), 4558]);
                        average180.push([new Date('2017/10/05'), 3, 2]);
                                        trade180.push([new Date('2017/10/05'), 6370]);
                        average180.push([new Date('2017/10/06'), 3, 2]);
                                        trade180.push([new Date('2017/10/06'), 6367]);
                        average180.push([new Date('2017/10/07'), 3, 2]);
                                        trade180.push([new Date('2017/10/07'), 8426]);
                        average180.push([new Date('2017/10/08'), 2, 2]);
                                        trade180.push([new Date('2017/10/08'), 708]);
                        average180.push([new Date('2017/10/09'), 2, 2]);
                                        trade180.push([new Date('2017/10/09'), 12188]);
                        average180.push([new Date('2017/10/10'), 1, 2]);
                                        trade180.push([new Date('2017/10/10'), 20242]);
                        average180.push([new Date('2017/10/11'), 1, 2]);
                                        trade180.push([new Date('2017/10/11'), 11125]);
                        average180.push([new Date('2017/10/12'), 1, 2]);
                                        trade180.push([new Date('2017/10/12'), 34358]);
                        average180.push([new Date('2017/10/13'), 1, 2]);
                                        trade180.push([new Date('2017/10/13'), 35474]);
                        average180.push([new Date('2017/10/14'), 1, 2]);
                                        trade180.push([new Date('2017/10/14'), 26955]);
                        average180.push([new Date('2017/10/15'), 1, 2]);
                                        trade180.push([new Date('2017/10/15'), 3752]);
                        average180.push([new Date('2017/10/16'), 1, 2]);
                                        trade180.push([new Date('2017/10/16'), 18256]);
                        average180.push([new Date('2017/10/17'), 2, 2]);
                                        trade180.push([new Date('2017/10/17'), 5256]);
                        average180.push([new Date('2017/10/18'), 2, 2]);
                                        trade180.push([new Date('2017/10/18'), 1730]);
                        average180.push([new Date('2017/10/19'), 2, 2]);
                                        trade180.push([new Date('2017/10/19'), 6854]);
                        average180.push([new Date('2017/10/20'), 2, 2]);
                                        trade180.push([new Date('2017/10/20'), 6503]);
                        average180.push([new Date('2017/10/21'), 2, 2]);
                                        trade180.push([new Date('2017/10/21'), 7769]);
                        average180.push([new Date('2017/10/22'), 2, 2]);
                                        trade180.push([new Date('2017/10/22'), 1262]);
                        average180.push([new Date('2017/10/23'), 2, 2]);
                                        trade180.push([new Date('2017/10/23'), 11694]);
                        average180.push([new Date('2017/10/24'), 1, 2]);
                                        trade180.push([new Date('2017/10/24'), 8248]);
                        average180.push([new Date('2017/10/25'), 2, 2]);
                                        trade180.push([new Date('2017/10/25'), 10440]);
                        average180.push([new Date('2017/10/26'), 3, 2]);
                                        trade180.push([new Date('2017/10/26'), 7733]);
                        average180.push([new Date('2017/10/27'), 3, 2]);
                                        trade180.push([new Date('2017/10/27'), 8758]);
                        average180.push([new Date('2017/10/28'), 3, 2]);
                                        trade180.push([new Date('2017/10/28'), 2183]);
                        average180.push([new Date('2017/10/29'), 3, 2]);
                                        trade180.push([new Date('2017/10/29'), 307]);
                        average180.push([new Date('2017/10/30'), 2, 2]);
                                        trade180.push([new Date('2017/10/30'), 8316]);
                        average180.push([new Date('2017/10/31'), 2, 2]);
                                        trade180.push([new Date('2017/10/31'), 2387]);
                        average180.push([new Date('2017/11/01'), 2, 2]);
                                        trade180.push([new Date('2017/11/01'), 3234]);
                        average180.push([new Date('2017/11/02'), 2, 2]);
                                        trade180.push([new Date('2017/11/02'), 7163]);
                        average180.push([new Date('2017/11/03'), 2, 1]);
                                        trade180.push([new Date('2017/11/03'), 4857]);
                        average180.push([new Date('2017/11/04'), 3, 1]);
                                        trade180.push([new Date('2017/11/04'), 8385]);
                        average180.push([new Date('2017/11/05'), 3, 1]);
                                        trade180.push([new Date('2017/11/05'), 1232]);
                        average180.push([new Date('2017/11/06'), 3, 1]);
                                        trade180.push([new Date('2017/11/06'), 3013]);
                        average180.push([new Date('2017/11/07'), 4, 2]);
                                        trade180.push([new Date('2017/11/07'), 2971]);
                        average180.push([new Date('2017/11/08'), 4, 2]);
                                        trade180.push([new Date('2017/11/08'), 6220]);
                        average180.push([new Date('2017/11/09'), 4, 2]);
                                        trade180.push([new Date('2017/11/09'), 1747]);
                        average180.push([new Date('2017/11/10'), 4, 2]);
                                        trade180.push([new Date('2017/11/10'), 5275]);
                        average180.push([new Date('2017/11/11'), 4, 2]);
                                        trade180.push([new Date('2017/11/11'), 3620]);
                        average180.push([new Date('2017/11/12'), 4, 2]);
                                        trade180.push([new Date('2017/11/12'), 297]);
                        average180.push([new Date('2017/11/13'), 3, 2]);
                                        trade180.push([new Date('2017/11/13'), 3625]);
                        average180.push([new Date('2017/11/14'), 3, 2]);
                                        trade180.push([new Date('2017/11/14'), 3293]);
                        average180.push([new Date('2017/11/15'), 3, 2]);
                                        trade180.push([new Date('2017/11/15'), 3737]);
                        average180.push([new Date('2017/11/16'), 3, 2]);
                                        trade180.push([new Date('2017/11/16'), 4211]);
                        average180.push([new Date('2017/11/17'), 3, 2]);
                                        trade180.push([new Date('2017/11/17'), 4643]);
                        average180.push([new Date('2017/11/18'), 3, 2]);
                                        trade180.push([new Date('2017/11/18'), 2548]);
                        average180.push([new Date('2017/11/19'), 3, 2]);
                                        trade180.push([new Date('2017/11/19'), 4982]);
                        average180.push([new Date('2017/11/20'), 3, 2]);
                                        trade180.push([new Date('2017/11/20'), 3492]);
                        average180.push([new Date('2017/11/21'), 3, 2]);
                                        trade180.push([new Date('2017/11/21'), 6281]);
                        average180.push([new Date('2017/11/22'), 3, 2]);
                                        trade180.push([new Date('2017/11/22'), 3290]);
                        average180.push([new Date('2017/11/23'), 3, 3]);
                                        trade180.push([new Date('2017/11/23'), 3971]);
                        average180.push([new Date('2017/11/24'), 3, 3]);
                                        trade180.push([new Date('2017/11/24'), 3568]);
                        average180.push([new Date('2017/11/25'), 3, 3]);
                                        trade180.push([new Date('2017/11/25'), 4940]);
                        average180.push([new Date('2017/11/26'), 3, 3]);
                                        trade180.push([new Date('2017/11/26'), 1039]);
                        average180.push([new Date('2017/11/27'), 3, 3]);
                                        trade180.push([new Date('2017/11/27'), 1574]);
                        average180.push([new Date('2017/11/28'), 3, 3]);
                                        trade180.push([new Date('2017/11/28'), 7132]);
                        average180.push([new Date('2017/11/29'), 3, 3]);
                                        trade180.push([new Date('2017/11/29'), 6625]);
                        average180.push([new Date('2017/11/30'), 3, 3]);
                                        trade180.push([new Date('2017/11/30'), 3159]);
                        average180.push([new Date('2017/12/01'), 3, 3]);
                                        trade180.push([new Date('2017/12/01'), 12181]);
                        average180.push([new Date('2017/12/02'), 3, 3]);
                                        trade180.push([new Date('2017/12/02'), 5952]);
                        average180.push([new Date('2017/12/03'), 3, 3]);
                                        trade180.push([new Date('2017/12/03'), 46]);
                        average180.push([new Date('2017/12/04'), 2, 3]);
                                        trade180.push([new Date('2017/12/04'), 3409]);
                        average180.push([new Date('2017/12/05'), 2, 3]);
                                        trade180.push([new Date('2017/12/05'), 10683]);
                        average180.push([new Date('2017/12/06'), 2, 3]);
                                        trade180.push([new Date('2017/12/06'), 518]);
                        average180.push([new Date('2017/12/07'), 3, 3]);
                                        trade180.push([new Date('2017/12/07'), 7765]);
                        average180.push([new Date('2017/12/08'), 3, 3]);
                                        trade180.push([new Date('2017/12/08'), 2893]);
                        average180.push([new Date('2017/12/09'), 3, 3]);
                                        trade180.push([new Date('2017/12/09'), 7434]);
                        average180.push([new Date('2017/12/10'), 3, 2]);
                                        trade180.push([new Date('2017/12/10'), 1010]);
                        average180.push([new Date('2017/12/11'), 3, 2]);
                                        trade180.push([new Date('2017/12/11'), 9703]);
                        average180.push([new Date('2017/12/12'), 2, 2]);
                                        trade180.push([new Date('2017/12/12'), 7709]);
                        average180.push([new Date('2017/12/13'), 2, 2]);
                                        trade180.push([new Date('2017/12/13'), 5017]);
                        average180.push([new Date('2017/12/14'), 3, 2]);
                                        trade180.push([new Date('2017/12/14'), 3664]);
                        average180.push([new Date('2017/12/15'), 3, 2]);
                                        trade180.push([new Date('2017/12/15'), 8062]);
                        average180.push([new Date('2017/12/16'), 3, 2]);
                                        trade180.push([new Date('2017/12/16'), 6200]);
                        average180.push([new Date('2017/12/17'), 3, 2]);
                                        trade180.push([new Date('2017/12/17'), 3795]);
                        average180.push([new Date('2017/12/18'), 4, 2]);
                                        trade180.push([new Date('2017/12/18'), 6141]);
                        average180.push([new Date('2017/12/19'), 5, 2]);
                                        trade180.push([new Date('2017/12/19'), 3670]);
                        average180.push([new Date('2017/12/20'), 5, 3]);
                                        trade180.push([new Date('2017/12/20'), 5359]);
                        average180.push([new Date('2017/12/21'), 5, 3]);
                                        trade180.push([new Date('2017/12/21'), 8826]);
                        average180.push([new Date('2017/12/22'), 5, 3]);
                                        trade180.push([new Date('2017/12/22'), 7256]);
                        average180.push([new Date('2017/12/23'), 5, 3]);
                                        trade180.push([new Date('2017/12/23'), 4241]);
                        average180.push([new Date('2017/12/24'), 6, 3]);
                                        trade180.push([new Date('2017/12/24'), 3912]);
                        average180.push([new Date('2017/12/25'), 6, 3]);
                                        trade180.push([new Date('2017/12/25'), 2923]);
                        average180.push([new Date('2017/12/26'), 6, 3]);
                                        trade180.push([new Date('2017/12/26'), 10002]);
                        average180.push([new Date('2017/12/27'), 5, 3]);
                                        trade180.push([new Date('2017/12/27'), 7760]);
                        average180.push([new Date('2017/12/28'), 5, 3]);
                                        trade180.push([new Date('2017/12/28'), 4760]);
                        average180.push([new Date('2017/12/29'), 5, 3]);
                                        trade180.push([new Date('2017/12/29'), 8245]);
                        average180.push([new Date('2017/12/30'), 5, 3]);
                                        trade180.push([new Date('2017/12/30'), 16014]);
                        average180.push([new Date('2017/12/31'), 5, 3]);
                                        trade180.push([new Date('2017/12/31'), 4475]);
                        average180.push([new Date('2018/01/01'), 5, 3]);
                                        trade180.push([new Date('2018/01/01'), 6562]);
                        average180.push([new Date('2018/01/02'), 5, 3]);
                                        trade180.push([new Date('2018/01/02'), 5610]);
                        average180.push([new Date('2018/01/03'), 5, 4]);
                                        trade180.push([new Date('2018/01/03'), 7359]);
                        average180.push([new Date('2018/01/04'), 6, 4]);
                                        trade180.push([new Date('2018/01/04'), 5222]);
                        average180.push([new Date('2018/01/05'), 6, 4]);
                                        trade180.push([new Date('2018/01/05'), 7991]);
                        average180.push([new Date('2018/01/06'), 6, 4]);
                                        trade180.push([new Date('2018/01/06'), 9680]);
                        average180.push([new Date('2018/01/07'), 6, 4]);
                                        trade180.push([new Date('2018/01/07'), 6059]);
                        average180.push([new Date('2018/01/08'), 6, 4]);
                                        trade180.push([new Date('2018/01/08'), 6093]);
                        average180.push([new Date('2018/01/09'), 6, 4]);
                                        trade180.push([new Date('2018/01/09'), 14197]);
                        average180.push([new Date('2018/01/10'), 5, 4]);
                                        trade180.push([new Date('2018/01/10'), 6972]);
                        average180.push([new Date('2018/01/11'), 4, 4]);
                                        trade180.push([new Date('2018/01/11'), 8424]);
                        average180.push([new Date('2018/01/12'), 4, 4]);
                                        trade180.push([new Date('2018/01/12'), 8803]);
                        average180.push([new Date('2018/01/13'), 4, 4]);
                                        trade180.push([new Date('2018/01/13'), 13273]);
                        average180.push([new Date('2018/01/14'), 4, 5]);
                                        trade180.push([new Date('2018/01/14'), 7609]);
                        average180.push([new Date('2018/01/15'), 4, 5]);
                                        trade180.push([new Date('2018/01/15'), 6227]);
                        average180.push([new Date('2018/01/16'), 4, 5]);
                                        trade180.push([new Date('2018/01/16'), 18825]);
                        average180.push([new Date('2018/01/17'), 3, 5]);
                                        trade180.push([new Date('2018/01/17'), 1439]);
                        average180.push([new Date('2018/01/18'), 3, 4]);
                                        trade180.push([new Date('2018/01/18'), 9854]);
                        average180.push([new Date('2018/01/19'), 3, 4]);
                                        trade180.push([new Date('2018/01/19'), 3525]);
                        average180.push([new Date('2018/01/20'), 3, 4]);
                                        trade180.push([new Date('2018/01/20'), 7759]);
                        average180.push([new Date('2018/01/21'), 3, 4]);
                                        trade180.push([new Date('2018/01/21'), 4282]);
                        average180.push([new Date('2018/01/22'), 4, 4]);
                                        trade180.push([new Date('2018/01/22'), 7970]);
                        average180.push([new Date('2018/01/23'), 3, 4]);
                                        trade180.push([new Date('2018/01/23'), 5571]);
                        average180.push([new Date('2018/01/24'), 3, 4]);
                                        trade180.push([new Date('2018/01/24'), 1825]);
                        average180.push([new Date('2018/01/25'), 3, 4]);
                                        trade180.push([new Date('2018/01/25'), 2877]);
                        average180.push([new Date('2018/01/26'), 4, 4]);
                                        trade180.push([new Date('2018/01/26'), 5258]);
                        average180.push([new Date('2018/01/27'), 4, 4]);
                                        trade180.push([new Date('2018/01/27'), 4483]);
                        average180.push([new Date('2018/01/28'), 4, 4]);
                                        trade180.push([new Date('2018/01/28'), 982]);
                        average180.push([new Date('2018/01/29'), 4, 4]);
                                        trade180.push([new Date('2018/01/29'), 9984]);
                        average180.push([new Date('2018/01/30'), 4, 4]);
                                        trade180.push([new Date('2018/01/30'), 1003]);
                        average180.push([new Date('2018/01/31'), 4, 4]);
                                        trade180.push([new Date('2018/01/31'), 8589]);
                        average180.push([new Date('2018/02/01'), 4, 4]);
                                        trade180.push([new Date('2018/02/01'), 11749]);
                        average180.push([new Date('2018/02/02'), 3, 4]);
                                        trade180.push([new Date('2018/02/02'), 17634]);
                        average180.push([new Date('2018/02/03'), 2, 4]);
                                        trade180.push([new Date('2018/02/03'), 25650]);
                        average180.push([new Date('2018/02/04'), 2, 3]);
                                        trade180.push([new Date('2018/02/04'), 1165]);
                        average180.push([new Date('2018/02/05'), 3, 3]);
                                        trade180.push([new Date('2018/02/05'), 5894]);
                        average180.push([new Date('2018/02/06'), 3, 3]);
                                        trade180.push([new Date('2018/02/06'), 17497]);
                        average180.push([new Date('2018/02/07'), 2, 3]);
                                        trade180.push([new Date('2018/02/07'), 7445]);
                        average180.push([new Date('2018/02/08'), 3, 3]);
                                        trade180.push([new Date('2018/02/08'), 3169]);
                        average180.push([new Date('2018/02/09'), 3, 3]);
                                        trade180.push([new Date('2018/02/09'), 12407]);
                        average180.push([new Date('2018/02/10'), 3, 3]);
                                        trade180.push([new Date('2018/02/10'), 2601]);
                        average180.push([new Date('2018/02/11'), 4, 3]);
                                        trade180.push([new Date('2018/02/11'), 7582]);
                        average180.push([new Date('2018/02/12'), 4, 3]);
                                        trade180.push([new Date('2018/02/12'), 506]);
                        average180.push([new Date('2018/02/13'), 5, 3]);
                                        trade180.push([new Date('2018/02/13'), 4643]);
                        average180.push([new Date('2018/02/14'), 5, 3]);
                                        trade180.push([new Date('2018/02/14'), 102]);
                        average180.push([new Date('2018/02/15'), 5, 3]);
                                        trade180.push([new Date('2018/02/15'), 10427]);
                        average180.push([new Date('2018/02/16'), 4, 3]);
                                        trade180.push([new Date('2018/02/16'), 7405]);
                        average180.push([new Date('2018/02/17'), 4, 3]);
                                        trade180.push([new Date('2018/02/17'), 7757]);
                        average180.push([new Date('2018/02/18'), 4, 3]);
                                        trade180.push([new Date('2018/02/18'), 7511]);
                        average180.push([new Date('2018/02/19'), 4, 3]);
                                        trade180.push([new Date('2018/02/19'), 9603]);
                        average180.push([new Date('2018/02/20'), 4, 3]);
                                        trade180.push([new Date('2018/02/20'), 3932]);
                        average180.push([new Date('2018/02/21'), 5, 3]);
                                        trade180.push([new Date('2018/02/21'), 1418]);
                        average180.push([new Date('2018/02/22'), 5, 3]);
                                        trade180.push([new Date('2018/02/22'), 8728]);
                        average180.push([new Date('2018/02/23'), 5, 3]);
                                        trade180.push([new Date('2018/02/23'), 7731]);
                        average180.push([new Date('2018/02/24'), 5, 3]);
                                        trade180.push([new Date('2018/02/24'), 2986]);
                        average180.push([new Date('2018/02/25'), 5, 3]);
                                        trade180.push([new Date('2018/02/25'), 7450]);
                        average180.push([new Date('2018/02/26'), 5, 3]);
                                        trade180.push([new Date('2018/02/26'), 7999]);
                        average180.push([new Date('2018/02/27'), 4, 3]);
                                        trade180.push([new Date('2018/02/27'), 14039]);
                        average180.push([new Date('2018/02/28'), 3, 3]);
                                        trade180.push([new Date('2018/02/28'), 5352]);
                        average180.push([new Date('2018/03/01'), 4, 3]);
                                        trade180.push([new Date('2018/03/01'), 3022]);
                        average180.push([new Date('2018/03/02'), 5, 3]);
                                        trade180.push([new Date('2018/03/02'), 11631]);
                        average180.push([new Date('2018/03/03'), 6, 3]);
                                        trade180.push([new Date('2018/03/03'), 12587]);
                        average180.push([new Date('2018/03/04'), 6, 4]);
                                        trade180.push([new Date('2018/03/04'), 2936]);
                        average180.push([new Date('2018/03/05'), 7, 4]);
                                        trade180.push([new Date('2018/03/05'), 7598]);
                        average180.push([new Date('2018/03/06'), 8, 4]);
                                        trade180.push([new Date('2018/03/06'), 11157]);
                        average180.push([new Date('2018/03/07'), 8, 4]);
                                        trade180.push([new Date('2018/03/07'), 6153]);
                        average180.push([new Date('2018/03/08'), 9, 4]);
                                        trade180.push([new Date('2018/03/08'), 1021]);
                        </script>
the average30,90,180 containers contain what information I need. In "average180.push([new Date('2018/03/08'), 9, 4]);" I would need the date and the 9 in this case, and I would need that for each call to a "average" container in this script. If anyone has experience with this that would be great!
Reply
#2
Use Selenium to scrape the actually executed data(what you see in browser) from JavaScript tag that you have parsed.
Can parse with Selenium of send data to Beautiful Soup for parsing.
After Selenium 3.0 both Chrome and FireFox has headless mode in there drivers,that said so do PhantomJS still work just get a deprecated message.
Reply
#3
Thank you for your reply I will check it out
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Weird characters scraped samuelbachorik 3 857 Oct-29-2023, 02:36 PM
Last Post: DeaD_EyE
  Web scraper not populating .txt with scraped data BlackHeart 5 1,458 Apr-03-2023, 05:12 PM
Last Post: snippsat
  Python Obstacles | Krav Maga | Wiki Scraped Content [Column Copy] BrandonKastning 4 2,163 Jan-03-2022, 06:59 AM
Last Post: BrandonKastning
  Python Obstacles | Kapap | Wiki Scraped Content [Column Nulling] BrandonKastning 2 1,690 Jan-03-2022, 04:26 AM
Last Post: BrandonKastning
  Beautiful Soup - access a rating value in a class KatMac 1 3,420 Apr-16-2021, 01:27 PM
Last Post: snippsat
  *Beginner* web scraping/Beautiful Soup help 7ken8 2 2,561 Jan-28-2021, 04:26 PM
Last Post: 7ken8
  Any way to remove HTML tags from scraped data? (I want text only) SeBz2020uk 1 3,414 Nov-02-2020, 08:12 PM
Last Post: Larz60+
  Help: Beautiful Soup - Parsing HTML table ironfelix717 2 2,623 Oct-01-2020, 02:19 PM
Last Post: snippsat
  cant loop through scraped site matt42 3 2,377 Aug-12-2020, 06:48 AM
Last Post: ndc85430
  Beautiful Soup (suddenly) doesn't get full webpage html j.crater 8 16,394 Jul-11-2020, 04:31 PM
Last Post: j.crater

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020