How to build a time range as a value from Pandas or MatPlotLib

Question

I have several DataFrames containing time series data, and would like to create a simple visualization of the range of time ranges for each of these DataFrames. Since I could not generate this using code, I included a sketch to illustrate my purpose.

Here is some code to create three DataFrames that are greatly simplified, smaller versions of the DataFrames I'm working with.

from pandas import DataFrame from numpy import datetime64, random # example data recorded by two different sensors example_data = random.rand(5,2) example_data2 = random.rand(9,2) example_data3 = random.rand(9,2) # timestamps from sensor1 times = ['2000-01-01 09:00:00', '2000-01-01 09:15:00', '2000-01-01 09:30:00', '2000-01-01 09:45:00', '2000-01-01 10:00:00'] # timestamps from sensor2 times2 = ['2000-01-01 08:45:00', '2000-01-01 09:00:00', '2000-01-01 09:15:00', '2000-01-01 09:30:00', '2000-01-01 09:45:00', '2000-01-01 10:00:00', '2000-01-01 10:15:00', '2000-01-01 10:30:00', '2000-01-01 10:45:00'] # timestamps from sensor2 times3 = ['2000-01-01 09:20:00', '2000-01-01 09:40:00', '2000-01-01 10:00:00', '2000-01-01 10:20:00', '2000-01-01 10:40:00', '2000-01-01 11:00:00', '2000-01-01 11:20:00', '2000-01-01 11:40:00', '2000-01-01 12:00:00'] # create the DataFrame object for sensor1 with the times and data above sensor1 = DataFrame({'Time': times, 'measure1': example_data[:,0], 'measure2': example_data[:,1]}) # create the DataFrame object for sensor2 with the times and data above sensor2 = DataFrame({'Time': times2, 'measure1': example_data2[:,0], 'measure2': example_data2[:,1]}) # create the DataFrame object for sensor2 with the times and data above sensor3 = DataFrame({'Time': times3, 'measure1': example_data3[:,0], 'measure2': example_data3[:,1]}) # coerce the 'Time' column from string to a numpy datetime64 value sensor1['Time'] = sensor1['Time'].astype(datetime64) sensor2['Time'] = sensor2['Time'].astype(datetime64) sensor3['Time'] = sensor3['Time'].astype(datetime64) 

I tried to take the min and max datetime values ​​from each of the DataFrames and put them in a new DataFrame, but when I try to build them, I get an error that there are no values ​​to build.

I also tried to take only the “Time” column and set Integer to the “value” column (that is, sensor 1 receives the broadcast of Int 1 into the column “value”, sensor 2 receives the broadcast of Int 2 and so on), then merges these DataFrames.

But this results in a lot of duplicate values ​​in the Time column and Nan values ​​in the value column.

I ran out of ideas on how to make this work.

EDIT: fixed 2001 timestamp in code block; -)

Answer 1

 import numpy import pandas # create an index containing all time stamps idx1 = pandas.Index(sensor1.Time) idx2 = pandas.Index(sensor2.Time) idx3 = pandas.Index(sensor3.Time) df = pandas.DataFrame(index=idx1.union(idx2).union(idx3)) # create a (constant) Series for each sensor df['Sensor1'] = df.index.to_series().apply(lambda x: 3 if x >= sensor1.Time.min() and x <= sensor1.Time.max() else numpy.NaN) df['Sensor2'] = df.index.to_series().apply(lambda x: 2 if x >= sensor2.Time.min() and x <= sensor2.Time.max() else numpy.NaN) df['Sensor3'] = df.index.to_series().apply(lambda x: 1 if x >= sensor3.Time.min() and x <= sensor3.Time.max() else numpy.NaN) # plot p = df.plot(ylim=[0, 4], legend=False) p.set_yticks([1., 2., 3.]) p.set_yticklabels(['Sensor3', 'Sensor2', 'Sensor1']) 

By the way, are you sure that you have the year 2001 at your timestamps? This will make your Sensor1 graph invisible small.

Answer 2

+1 for hand-drawn graphics. I like to run my visualization code using a drawn graph to determine the goal, so I don't get lateral tracking with formatting or what if scanarios.