\section{Introduction}
\label{sec:introduction}

The Magnuson-Stevens Fishery Conservation and Management Act~\cite{msfcact}, the framework for fisheries management in the United States, requires that managed fish stocks undergo periodic assessment to determine if they are overfished or are experiencing overfishing. A basic stock assessment requires data on fishery abundance, biology (e.g. age, growth, fecundity), and catch. While demands to continually improve stock assessments are high, the greatest impediment to their accuracy, precision, and credibility remains a lack of adequate input data \cite{deriso1998improving}. Recent developments in low-cost autonomous underwater vehicles (AUVs), stationary camera arrays, and towed vehicles has made it possible for fishery scientists to begin generating species-specific, size-structured abundance estimates for different species of marine organisms from imagery and video (see Figure~\ref{fig:platforms}). To this end, NOAA Fisheries and other agencies are increasingly employing camera-based surveys for abundance estimation \cite{cappo2006counting}. However, the volume of optical data produced quickly exceeds the capabilities of human analysis. To move into operational use, automated video analysis solutions are needed to extract species-specific, size-structured abundance measures from optical data streams.

\begin{figure}[h]
  \centering
  \includegraphics[width=\linewidth]{content/figures/platforms_ex.png}
  \caption{A few examples of devices used to collect imagery underwater, 
  including towed vehicles (top right~\cite{williams2010cam}, bottom right~\cite{gallager2004high})
  and a stationary camera array (left). }
  \label{fig:platforms}
\end{figure}

This paper presents an open-source computer vision software platform for automating the image analysis process, for applications in marine imagery or any other type of video analytics. The system provides a common interface for several algorithm stages (stereo matching, object detection, object recognition), multiple implementations of each, as well as unified methods for scoring different algorithms for accomplishing the same task. The common open-source framework facilitates the development of additional image analysis modules and pipelines through continuing collaboration within the image analysis and fisheries science communities.

\begin{figure}[h]
  \centering
  \includegraphics[width=\linewidth]{content/figures/data_ex.png}
  \caption{Example optical data from the devices shown in Figure~\ref{fig:platforms}.}
  \label{fig:data}
\end{figure}

The platform can be divided into three core components: the pipeline processing framework and infrastructure; image processing elements that fit into the framework; and auxiliary tools outside the streaming framework that provide training, GUIs and scoring. The pipeline subsystem allows image processing elements to be implemented in the most popular languages used for computer vision, including C, C++, Python, and Matlab. A graphical interface is provided in the framework for visualizing individual object detections, making new annotations, and filtering detections based on classification values. There are two separate scoring tools included, one for generating basic statistics over detections compared with groundtruth (detection rates, specificity, false alarm rate, etc.) and a second for generating receiver operating characteristic (ROC) curves for detections which contain associated category probabilities. Both the scoring and GUI tools work with either single frame object detections, or spatiotemporal object tracks.

The platform has been released as open-source software on github. Multiple object detectors specialized to different types of marine imagery have been integrated, with results shown in Section~\ref{sec:modules}. The provided scoring elements are also available, with example computed metrics and ROC curves for selected detection problems shown in Section~\ref{sec:scoring}. It is anticipated that the NOAA community will continue to adopt the framework, enabling collaboration and re-use of modules across different research groups working on related problems. Although the framework is primarily being used for marine imagery at present, it is based on a more generic pipelined vision processing system that applies to video analytics in any domain. It is hoped that the general vision community would find the framework useful and that a vibrant open-source community will develop around the platform.
