Sometimes batch items are completely nan
Sometimes a train batch item is completely nan. These examples should be filtered out with the current settings, but it seems they still get through sometime. We need to determine why this is the case and fix it.
Example:
The relevant config settings that should prevent the following example are:
data:
observable_threshold : 0.2
resample_invalid_frames : 3
force_bad_frames: False
Tasks:
-
Reproducible MWE -
Add information to the visualization to indicate when / where augmentation is applied / which frames were requested / resampled, number dropped frames / etc... -
Where is observable threshold / resample invalid frames failing? -
A fix
Edited by Jon Crall