Feature: Sharding
One strategy for handling larger datasets or distributed datasets would be sharding. We already have a method for doing a union of two kwcoco files, it would be useful if there was a mechanism to store data for a single dataset across multiple files, i.e. one or more files for all of the images, and then one ore more file for each annotations. In the limit, we could have one annotation file per image. One challenge is that annotations need to reference images and category ids. We would either need to duplicate image / category information or provide a way to reference it.