VTKCharts: vtkPlotLine cannot visualize data upto nanosecond precision
At work, we'd like to use VTK charts API to plot time signals with nanosecond precision. Upon comparison and benchmarking against matplotlib, vispy and gnuplot, the abstraction, functionality, features of VTK Charts seem superior, all without a compromise on performance.
Only one hiccup though, the timestamps are of type int64_t
from Linux clock_gettime().
For example, in python:
>>> ts = time.time_ns()
>>> ts
1620377676274264418 # our X-Axis data is typically long 64-bit integers like these.
>>> pd.to_datetime(ts)
Timestamp('2021-05-07 08:54:36.274264418')
There is a similar issue #17845 that brings up a limitation of Charts API to 32-bit floats. Even if that were fixed, it is insufficient. The least significant 10-11 bits from the 64-bit timestamp are lost in the conversion to 64-bit float value.
8 double c = static_cast<double>(~0ul >> 1);
(gdb) p (~0ul >> 1)
$2 = 9223372036854775807
(gdb) printf "%1.18f\n", c
9223372036854775808.000000000000000000
I think a subclass of vtkPlotLine, specialized to deal with time precision signals is necessary here. (Matplotlib has ConciseDateFormatter for that purpose.)
In the example, the X-axis values are regularly spaced (interval is 8 nanoseconds)
[1536852149738895586 1536852149738895594 1536852149738895602
....
1536852149738896162 1536852149738896170 1536852149738896178
1536852149738896186]
If I plot the 64-bit data simply, the VTK plot is unrealistic of the data. It naively casts away the lower bits, so there is a jump.
Whereas, if I reinterpret the x-data as 32-bit signed integers and remove the higher-order bits, then I get something useful.
In C++, it'd be
auto new_x = reinterpret_cast<int32_t*>(&x)
// stride such that it skips higher order bits..
In python, I reinterpret and stride with numpy
# xs = numpy_support.numpy_to_vtk(dobj.xdata)
xs = numpy_support.numpy_to_vtk(dobj.xdata.view(np.int32)[::2])
With that change, I get a proper representation of the signal, but with only the lower 32 bits.
I found the sources in Charts/Core, Rendering/Context2D and Rendering/ContextOpenGL2. I'll try to implement such date-time functionality in Charts/Core and update Common/Core/vtkTimePointUtility to support nanosecond timestamps.
In a subclass of vtkPlotLine
.
- Require that the x-data is of type int64 or uint64.
- Charts API is limited to 32-bit floats. So try to shave off unnecessary information from the data points. If the data range is in the same year, discard those digits. Repeat the same for subsequent periods (month, day, hour, minute, seconds, nanoseconds) until one of those varies. This sort of dynamic time-based resolution mechanism could observe a UpdateRange event from the X-Axis.
- Generate iso-format tick labels from 64-bit time stamps and position them nicely (Remove redundant years, months, days.., rotate the tick labels if necessary)
- IT'd be nice if the plot tooltip labels include entire timestamp information.