Add ability to "Trimap" annotations
This is an idea from @jacobsn based on https://github.com/lnugraha/trimap_generator
The idea is that we are given annotations that are conservative in ther spacetime extent, and we want to expand a large buffer zone around them which effectively corresponds to an "ignore" region.
I have something like this implemented already, via a paramter currently called "dist_weights", which has a help section that currently reads:
dist_weights = scfg.Value(0, group=WEIGHT_GROUP, help=ub.paragraph(
'''
To use distance-transform based weights on annotations or not
'''))
An example that illustrates its usage is:
def demo_dataloader_dilate():
import geowatch
from geowatch.tasks.fusion.datamodules.kwcoco_dataset import KWCocoVideoDataset
coco_dset = geowatch.coerce_kwcoco('vidshapes1', num_frames=10)
self = KWCocoVideoDataset(coco_dset, time_dims=5,
window_dims=(512, 512),
balance_areas=True,
dist_weights=True,
normalize_perframe=False)
self.disable_augmenter = True
# Choose the target region we will sample
target = self.new_sample_grid['targets'][self.new_sample_grid['positives_indexes'][0]]
target['dist_weights'] = True
item1 = self[target]
target['dist_weights'] = False
item2 = self[target]
# Visualize the item
canvas1 = self.draw_item(item1)
canvas2 = self.draw_item(item2)
import kwplot
kwplot.autompl()
kwplot.imshow(canvas1, pnum=(1, 2, 1), fnum=1, title='dist_weights=1')
kwplot.imshow(canvas2, pnum=(1, 2, 2), fnum=1, title='dist_weights=0')
kwplot.show_if_requested()
Which produces the plot:
There is also a similar ignore_dilate
and weight_dilate
parmeter:
ignore_dilate = scfg.Value(0, group=WEIGHT_GROUP, help='Dilation applied to ignore masks.')
weight_dilate = scfg.Value(0, group=WEIGHT_GROUP, help='Dilation applied to weight masks.')
Which just blingly applies dilation to ignore masks or weights respectively.
I think the idea here would be to add a parameter that to specify how big a region around an annotation should be considered "uncertain", which would correspond to a dilation, and then mark that area as "ignore".
The implementation detail which makes this somewhat challenging is that when we sample a window, we currently only return annotations that intersect with that window, so if there was a small annotation next a target window as in this illustration:
The current implementation would say: "there is no annotation that intersects this window so I will return no annotations for you to work with". That is a problem, because we need to be able to mark the uncertainty region that exists within the target bounds, but we can't do that unless we know there is an annotation there. Thus we could do two things:
- update ndsampler to search for a buffer region around the sampled target for annotations (This is probably the best method).
- annotate the entire buffer region and then errode it to the part of the annotation we should be confident in. (I like this idea less).
@jacobsn thoughts? Do you have anyone that would be interested in adding this feature?