Python Forum
Visualisation of gaps in time series data
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Visualisation of gaps in time series data
#11
Thanks, that worked!!! My plot now looks like this:

[Image: zUcgC]

Is it maybe possible to display those dates on the x-axis which start and end a data gap? So that one can easily identify these gaps? Because now it seems that they were randomly chosen.

My code looks like this now:

import pandas as pd
from matplotlib import pyplot as plt
import numpy as np

colnames=['Timestamp','Currency','Rate','Volume']
usecols=['Timestamp','Currency']
series=pd.read_csv('prices.csv', names=colnames, usecols=usecols)

series.Timestamp = pd.to_datetime(series.Timestamp)

series = series.set_index('Timestamp')

missing = series.resample('3h').min().notnull()
missing = missing[missing.columns[0]].astype(int)

plt.bar(missing.index, np.ones(len(missing)), color = [['y','b'][idx] for idx in missing], width = 1)
plt.xticks(rotation=70)
Reply
#12
I think that matplotlib try to put some small number of equidistantly spaced xticks (likely it depends on figure size and dpi)

Its possible to use
plt.xticks(list_of_x_values)
to put xticks exactly where you want. And its not too hard to get starting and ending timestamps. If you have timeserie "missing" with values indicating missing values (False=missing), you can use something like
starts = missing.index[~missing & missing.shift(1).fillna(False)]
ends = missing.index[~missing & missing.shift(-1).fillna(False)]
to get starting and ending timestamps of gaps. You can try to plot these timestamps as your xticks with
plt.xticks(np.concatenate([starts, ends]))
Unfortunately there is high probability that for "real" data you will end with unreadable labels; you need to have some "space" between labels, so you would likely need to "preprocess" these timestamps and select only some of them (something like "big gaps" only, but you need to take care if there are no "big gaps" ... Morever you probably would need to check spaces between end of one gap and start of next one).
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Fit straight line to pandas time series data with semilog plot schniefen 2 1,500 Mar-10-2023, 01:08 PM
Last Post: jefsummers
  Plot time series data schniefen 3 1,270 Mar-04-2023, 04:22 PM
Last Post: noisefloor
  Help on Time Series problem Kishore_Bill 1 4,775 Feb-27-2020, 09:07 AM
Last Post: Kishore_Bill
  10fold cross-validation on time series ulrich48155 5 9,138 May-08-2017, 04:36 PM
Last Post: ulrich48155

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020