Skip to content

Add partitions processing

Alexandre Boyer requested to merge add-partition-feature into master

Summary

This MR adds a new CLI argument --partitions to download specific videos from a given set of corpus included in given partitions.

Main

  • Split actev-corpora-maint.py into several files:
scripts/actev-corpora-maint.py
scripts/actev-corpora-maint/corpus.py
scripts/actev-corpora-maint/partition.py
scripts/actev-corpora-maint/utils.py
  • Add a new class Partition
scripts/actev-corpora-maint/partition.py
  • Enhance the Corpus.load_corpus method to manage the partitions

CLI help

Here is a new way to only download the ActEV-Eval-CLI-Validation-Set1/2 videos from the MEVA corpus:

MEVA videos included in each partition:

  • ActEV-Eval-CLI-Validation-Set1: 2018-03-14.07-30-04.07-35-04.school.G336.avi
  • ActEV-Eval-CLI-Validation-Set2: 2018-03-14.07-30-04.07-35-04.school.G336.avi, 2018-03-15.15-55-01.16-00-01.admin.G326.avi

The given partitions also include 4 videos of the VIRAT-V2 corpus.

$ python3 scripts/actev-corpora-maint.py --operation download --corpus MEVA --partitions ActEV-Eval-CLI-Validation-Set1 ActEV-Eval-CLI-Validation-Set2

Downloading 2018-03-14.07-30-04.07-35-04.school.G336.avi
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   162  100   162    0     0   3291      0 --:--:-- --:--:-- --:--:--  3306
Downloading 2018-03-15.15-55-01.16-00-01.admin.G326.avi
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   162  100   162    0     0   3291      0 --:--:-- --:--:-- --:--:--  3306
  • Note: 2018-03-14.07-30-04.07-35-04.school.G336.avi appears in both partition but is only processed once. *

Checklist

  • Includes a summary of the purpose of the MR and commits
  • Link to any issues that this MR addresses
  • Includes tests
  • (if needed) documentation was updated
  • [When ready for code review] Added label Review (to both issue and MR)
  • Code reviewer assigned to merge request
Edited by Alexandre Boyer

Merge request reports