Off by one bug - align corners issue.
In tests/test_off_by_one.py :: test_off_by_one_with_small_img there is a test that demonstrates a failure case (or at least an unititive case in delayed image).
The following code
import delayed_image
import kwimage
import numpy as np
raw = np.linspace(0, 1, 36).reshape(6, 6)
delayed = delayed_image.DelayedIdentity(raw)
warp = kwimage.Affine.coerce(offset=(-1e-8, -1e-9), scale=(8.6, 8.5))
x = delayed.warp(warp)
x.finalize(interpolation='nearest')
warp = kwimage.Affine.coerce(offset=(0, 0), scale=(8.6, 8.5))
x = delayed.warp(warp)
data = x.finalize(interpolation='nearest')
warp = kwimage.Affine.coerce(offset=(0, 0), scale=(2, 2))
warp = warp @ kwimage.Affine.translate((0.5, 0.5))
x = delayed.warp(warp)
data2 = x.finalize(interpolation='nearest')
data3 = kwimage.imresize(raw, scale=2.0, interpolation='nearest')
if 1:
import kwplot
pnum_ = kwplot.PlotNums(nRows=1, nCols=4)
kwplot.autompl()
kwplot.plt.ion()
kwplot.imshow(raw, pnum=pnum_())
kwplot.imshow(kwimage.fill_nans_with_checkers(data), pnum=pnum_())
kwplot.imshow(kwimage.fill_nans_with_checkers(data2), pnum=pnum_())
kwplot.imshow(kwimage.fill_nans_with_checkers(data3), pnum=pnum_())
The nan pixels on the bottom and right borders are undesirable. I think this has to do with an align corners issue. This needs a proper description and specification on how delayed image should be handling these cases. We should also get a better MWE to show the effect of this in practice when you have an output space that is much smaller than your input space.
See references for futher details on align corners issues:
- https://jricheimer.github.io/tensorflow/2019/02/11/resize-confusion/
- https://github.com/ClementPinard/SfmLearner-Pytorch/blob/master/inverse_warp.py
- https://discuss.pytorch.org/t/affine-transformation-matrix-paramters-conversion/19522
- https://github.com/pytorch/pytorch/issues/15386
- https://github.com/pytorch/pytorch/issues/20785
- https://github.com/pytorch/pytorch/pull/23923
- https://github.com/pytorch/pytorch/pull/24929
- https://user-images.githubusercontent.com/9757500/58150486-c5315900-7c34-11e9-9466-24f2bd431fa4.png