Commit aef4aa0a authored by Alexandre Boyer's avatar Alexandre Boyer
Browse files

Update documentation

parent aa5390a8
Pipeline #125123 passed with stage
in 28 seconds
......@@ -9,7 +9,7 @@ This repository contains:
You should fork the project and use the `master` branch in order to implement the entry point methods and get the updates.
Installation
Test it
============
Get the CLI
......@@ -22,8 +22,8 @@ Clone the repository:
git clone https://gitlab.kitware.com/alexandreB/diva_evaluation_cli.git
```
Automatic installation
----------------------
Install it
----------
* Go into the clone of this repository:
......@@ -64,15 +64,43 @@ $ actev -h
Documentation
-------------
Consult the documentation to have information: [cli](doc/cli.md)
Consult the documentation to have information: [CLI](doc/cli_commands.md)
Development
===========
Fork it and develop your own implementation
===========================================
The cli components are included in the `bin` directory, you do not have to modify it.
Fork it
-------
Click on the “Fork” button to make a copy of the repository in your own space and add this repository as a remote upstream to get the latest updates
```
$ git remote add upstream https://gitlab.kitware.com/alexandreB/diva_evaluation_cli.git
```
Update it
---------
Execute the following commands to update it:
```
$ git fetch upstream
$ git checkout master
$ git rebase upstream/master
```
Develop it
----------
The CLI components are included in the `bin` directory, you do not have to modify it.
To add your code, you simply have to implement the methods in `src/entry_points`.
We suggest you to call your scripts in the entry point methods and store them in `src`.
More information about the development and the update of the CLI here: [development](doc/development.md)
Contact
=======
* alexandre.boyer@nist.gov
* maxime.hubert@nist.gov
# actev design-chunks
## Description
Given a file index and an activity index, produce a chunks file that is suitable for the system.
The command contains the following subcommands:
## Parameters
| Name | Id | Required | Definition |
|-----------|-----|-----------------|----------------------------|
| file-index | f | True | path to file index json file |
| activity-index | a | True | path to activity index json file |
| output | o | True | path to save chunks file |
| nb-videos-per-chunk | n | False | number of videos in the chunk |
## Usage
Generic command:
```
actev design-chunks -f <path to file.json> -a <path to activity.json> -o <path to output result> -n <number of video per chunks>
```
Example:
```
actev design-chunks -f ~/file.json -a ~/activity.json -o ~/chunks.json -n 96
```
# actev exec
## Description
Calls a team-implemented API. Captures time stamps, resource usage, etc.
The command contains the following subcommands:
## Parameters
| Name | Id | Required | Definition |
|-----------|------------|---------|----------------------------|
| file-index | f | True | path to file index json file |
| activity-index | a | True | path to activity index json file |
| chunks | c | True | path to chunks json file |
| nb-video-per-chunk | n | False | number of videos in the chunk |
| video-location | v | True | path to videos content |
| system-cache-dir | s | True | path to system cache directory |
| config-file | C | False | path to config file |
| output-file | o | True | path to merge chunks command result|
| chunk_result | r | True | path to chunks json file after merge chunks execution |
## Usage
Generic command:
```
actev exec -f <path to file.json> -a <path to activity.json> -c <path to chunks.json > -n <number of videos per chunk> -v <path to videos directory> -s <path to system cache directory> -C <path to config file.json> -o <path to output.json> -r <path to chunks result>
```
Example:
```
actev exec -f ~/file.json -a ~/activity.json -c ~/chunks.json -n 4 -v ~/videos/ -s ~/system_cache_dir/ -o ~/output.json -r ~/chunks_result.json
```
# actev experiment-cleanup
## Description
Close any servers, terminates cluster (future functionality), etc.
The command contains the following subcommands:
## Usage
```
actev experiment-cleanup
```
\ No newline at end of file
# actev experiment-init
## Description
Start servers, starts cluster, etc.
The command contains the following subcommands:
## Parameters
| Name | Id | Required | Definition |
|-----------|------------|---------|----------------------------|
| file-index | f | True | path to file index json file |
| activity-index | a | True | path to activity index json file |
| chunks | c | True | path to chunks json file |
| nb-video-per-chunk | n | False | number of videos in the chunk |
| video-location | v | True | path to videos content |
| system-cache-dir | s | True | path to system cache directory |
| config-file | C | False | path to config file |
## Usage
Generic command:
```
actev experiment-init -f <path to file.json> -a <path to activity.json> -c <path to chunks.json > -v <path to videos directory> -s <path to system cache directory>
```
Example:
```
actev experiment-init -f ~/file.json -a ~/activity.json -c ~/chunks.json -v ~/videos/ -s ~/system_cache_dir/
```
CLI usage
=========
# actev get-system
Get the existing commands
-------------------------
## Description
```
actev -h
```
Get system
-------------
Downloads a credentialed, web-accessible content into `/src`.
```
actev get-system -h
```
The command contains the following subcommands:
### Docker
Generic command:
```
actev get-system docker -u <docker image url>
```
## get-system git
### Description
With username and password:
Clones a git repository
```
actev get-system docker -u <docker image url> -U <username> -p <password>
```
### Parameters
:warning: if your password starts with *-* as in this example: `-9r45ijFo0`, you should write `--password=-9r45ijFo0` instead of `-p -9r45ijFo0`. Otherwise, the value will be interpreted as an argument to parse.
| Name | Id | Required | Definition |
|-----------|----|-----------------|----------------------------|
| url | u | True | url to get the system |
| location | l | False | path to store the system |
| user | U | False | username to access the url |
| password | p | False | password to access the url |
| token | t | False | token to access the url |
Example:
```
actev get-system docker -u gitlab.kitware.com:4567/diva-baseline/diva-baseline:eval_cli
```
### Git
### Usage
Generic command:
......@@ -74,202 +58,89 @@ Example:
actev get-system git -u https://gitlab.kitware.com/alexandreB/diva_evaluation_cli.git -l /tmp
```
### Other (web resources as a tar file)
Generic command:
```
actev get-system other -u <web resource url>
```
With username and password:
```
actev get-system other -u <web resource url> -U <username> -p <password>
```
With a token:
```
actev get-system git -u <web resource url> -t <token>
```
Store the system in a specific directory:
```
actev get-system git -u <web resource url> ... -l <location>
```
:information_source: You can also directly add your credentials inside the url.
:warning: if your password or token starts with *-* as in this example: `-9r45ijFo0`, you should write `--password=-9r45ijFo0` instead of `-p -9r45ijFo0`. Otherwise, the value will be interpreted as an argument to parse.
Example:
```
actev get-system other -u https://path/to/file.tgz -l /tmp
```
System setup
------------
```
actev system-setup
```
Design chunks
-------------
Generic command:
```
actev design-chunks -f <path to file.json> -a <path to activity.json> -o <path to output result> -n <number of video per chunks>
```
Example:
```
actev design-chunks -f ~/file.json -a ~/activity.json -o ~/chunks.json -n 96
```
## get-system docker
### Description
Experiment initialization
-------------------------
Generic command:
Downloads a docker image
```
actev experiment-init -f <path to file.json> -a <path to activity.json> -c <path to chunks.json > -v <path to frames directory> -s <path to system cache directory>
```
### Parameters
Example:
| Name | Id | Required | Definition |
|-----------|-----|----------------|----------------------------|
| url | u | True | url to get the system |
| user | U | False | username to access the url |
| password | p | False | password to access the url |
```
actev experiment-init -f ~/file.json -a ~/activity.json -c ~/chunks.json -v ~/frames_location/ -s ~/system_cache_dir/
```
### Usage
Pre-process a chunk
-------------------
Generic command:
```
actev pre-process-chunk -i <id of a chunk>
actev get-system docker -u <docker image url>
```
Example:
With username and password:
```
actev pre-process-chunk -i Chunk1
actev get-system docker -u <docker image url> -U <username> -p <password>
```
Process a chunk
---------------
Generic command:
```
actev process-chunk -i <id of a chunk>
```
:warning: if your password starts with *-* as in this example: `-9r45ijFo0`, you should write `--password=-9r45ijFo0` instead of `-p -9r45ijFo0`. Otherwise, the value will be interpreted as an argument to parse.
Example:
```
actev process-chunk -i Chunk1
actev get-system docker -u gitlab.kitware.com:4567/diva-baseline/diva-baseline:eval_cli
```
Post process a chunk
--------------------
Generic command:
## get-system other
### Description
```
actev post-process-chunk -i <id of a chunk>
```
Downloads a web resources as a zip/tar file
Example:
### Parameters
```
actev post-process-chunk -i Chunk1
```
| Name | Id | Required | Definition |
|-----------|----|-----------------|----------------------------|
| url | u | True | url to get the system |
| location | l | False | path to store the system |
| user | U | False | username to access the url |
| password | p | False | password to access the url |
| token | t | False | token to access the url |
Reset a chunk
-------------
Generic command:
### Usage
```
actev reset-chunk -i <id of a chunk>
```
Example:
```
actev reset-chunk -i Chunk1
```
Merge chunks
------------
Generic command:
```
actev merge-chunks -o <path to merging result> -c <path to chunks file generated during merging> -r ~/<result directory> -i <list of chunk ids>
```
Example:
```
actev merge-chunks -o ~/output.json -c ~/merging_chunks.json -r ~/system_cache_dir/ -i Chunk1 Chunk2
```
Experiment cleanup
------------------
```
actev experiment-cleanup
```
Status
------
```
actev status -h
actev get-system other -u <web resource url>
```
### System query
With username and password:
```
actev status system-query
actev get-system other -u <web resource url> -U <username> -p <password>
```
### Experiment query
With a token:
```
actev experiment-query
actev get-system git -u <web resource url> -t <token>
```
### Chunk query
Generic command:
Store the system in a specific directory:
```
actev chunk-query -i <chunk id >
actev get-system git -u <web resource url> ... -l <location>
```
Example:
```
actev chunk-query -i Chunk1
```
:information_source: You can also directly add your credentials inside the url.
Validate and score an output
----------------------------
Generic command:
:warning: if your password or token starts with *-* as in this example: `-9r45ijFo0`, you should write `--password=-9r45ijFo0` instead of `-p -9r45ijFo0`. Otherwise, the value will be interpreted as an argument to parse.
```
actev validate-execution -o <path to output result> -r <path to reference file> -a <path to activity> -f <path to file> -R <path to scoring result>
```
Example:
```
actev validate-execution ~/output.json -r diva_evaluation_cli/container_output/dataset/output.json -a ~/activity.json -f ~/file.json -R ~/result.json
actev get-system other -u https://path/to/file.tgz -l /tmp
```
# actev merge-chunks
## Description
Given a list of chunk ids, merges all the chunks system output present in the list.
The command contains the following subcommands:
## Parameters
| Name | Id | Required | Definition |
|-----------|------------|---------|----------------------------|
| result-location | r | True | path to get the result of the chunks processing |
| chunk-file | c | True | path to directory where intermediate system output is stored |
| output-file | o | True | path to the output file generated |
| chunk-ids | i | False | list of chunk ids |
## Usage
Generic command:
```
actev merge-chunks -o <path to merging result> -c <path to chunks file generated during merging> -r ~/<result directory> -i <list of chunk ids>
```
Example:
```
actev merge-chunks -o ~/output.json -c ~/merging_chunks.json -r ~/system_cache_dir/ -i Chunk1 Chunk2
```
\ No newline at end of file
# actev post-process-chunk
## Description
Post process a chunk
The command contains the following subcommands:
## Parameters
| Name | Id | Required | Definition |
|-----------|------------|---------|----------------------------|
| chunk-id | i | True | chunk id |
| system-cache-dir | s | False | path to system cache directory |
## Usage
Generic command:
```
actev post-process-chunk -i <id of a chunk> -s <path to system cache directory>
```
Example:
```
actev post-process-chunk -i Chunk1 -s ~/system_cache_dir
```
\ No newline at end of file
# actev pre-process-chunk
## Description
Pre process a chunk
The command contains the following subcommands:
## Parameters
| Name | Id | Required | Definition |
|-----------|------------|---------|----------------------------|
| chunk-id | i | True | chunk id |
| system-cache-dir | s | False | path to system cache directory |
## Usage
Generic command:
```
actev pre-process-chunk -i <id of a chunk> -s <path to system cache directory>
```
Example:
```
actev pre-process-chunk -i Chunk1 -s ~/system_cache_dir
```
\ No newline at end of file
# actev process-chunk
## Description
Process a chunk
The command contains the following subcommands:
## Parameters
| Name | Id | Required | Definition |
|-----------|------------|---------|----------------------------|
| chunk-id | i | True | chunk id |
| system-cache-dir | s | False | path to system cache directory |
## Usage
Generic command:
```
actev process-chunk -i <id of a chunk> -s <path to system cache directory>
```
Example:
```
actev process-chunk -i Chunk1 -s ~/system_cache_dir
```
\ No newline at end of file
# actev reset-chunk
## Description
Reset a chunk
The command contains the following subcommands:
## Parameters
| Name | Id | Required | Definition |
|-----------|------------|---------|----------------------------|
| chunk-id | i | True | chunk id |
| system-cache-dir | s | False | path to system cache directory |
## Usage
Generic command:
```
actev reset-chunk -i <id of a chunk> -s <path to system cache directory>
```
Example:
```