% -*- latex -*-

\chapter{Managing Devices}
\label{chap:ManagingDevices}

Multiple vendors vie to provide accelerator-type processors.
\VTKm endeavors to support as many such architectures as possible.
Each device and device technology requires some level of code specialization, and that specialization is encapsulated in a unit called a \index{device adapter}\keyterm{device adapter}.

So far in Part \ref{part:Using} we have been writing code that runs on a local serial CPU.
In those examples where we run a filter, \VTKm is launching parallel execution in the execution environment.
Internally \VTKm uses a device adapter to manage this execution.

A build of \VTKm generally supports multiple device adapters.
In this chapter we describe how to represent and manage devices.


\section{Device Adapter Tag}
\label{sec:DeviceAdapterTag}

\index{device adapter!tag|(}
\index{tag!device adapter|(}

A device adapter is identified by a \keyterm{device adapter tag}.
This tag, which is simply an empty struct type, is used as the template parameter for several classes in the \VTKm control environment and causes these classes to direct their work to a particular device.
The following device adapter tags are available in \VTKm.

\index{device adapter!tag!provided|(}
\index{tag!device adapter!provided|(}

\begin{description}
\item[\vtkmcont{DeviceAdapterTagSerial}] \index{serial}
  Performs all computation on the same single thread as the control environment.
  This device is useful for debugging.
  This device is always available.
  This tag is defined in \vtkmheader{vtkm/cont}{DeviceAdapterSerial.h}.
\item[\vtkmcont{DeviceAdapterTagCuda}] \index{CUDA}
  Uses a CUDA capable GPU device.
  For this device to work, VTK-m must be configured to use CUDA and the code must be compiled by the CUDA \textfilename{nvcc} compiler.
  This tag is defined in \vtkmheader{vtkm/cont/cuda}{DeviceAdapterCuda.h}.
\item[\vtkmcont{DeviceAdapterTagOpenMP}] \index{OpenMP}
  Uses OpenMP compiler extensions to run algorithms on multiple threads.
  For this device to work, VTK-m must be configured to use OpenMP and the code must be compiled with a compiler that supports OpenMP pragmas.
  This tag is defined in \vtkmheader{vtkm/cont/openmp}{DeviceAdapterOpenMP.h}.
\item[\vtkmcont{DeviceAdapterTagTBB}] \index{Intel Threading Building Blocks} \index{TBB}
  Uses the Intel Threading Building Blocks library to run algorithms on multiple threads.
  For this device to work, VTK-m must be configured to use TBB and the executable must be linked to the TBB library.
  This tag is defined in \vtkmheader{vtkm/cont/tbb}{DeviceAdapterTBB.h}.
\item[\vtkmcont{DeviceAdapterTagKokkos}] \index{Kokkos}
  Uses the Kokkos library to run algorithms in parallel.
  For this device to work, VTK-m must be configured to use Kokkos and the executable must be linked to the Kokkos libraries.
  Vtk-m will use the default execution space of the provided kokkos library build.
  This tag is defined in \vtkmheader{vtkm/cont/kokkos}{DeviceAdapterKokkos.h}.
\end{description}

\index{tag!device adapter!provided|)}
\index{device adapter!tag!provided|)}

The following example uses the tag for the Intel Threading Building blocks device adapter to specify a specific device for \VTKm to use.
(Details on specifying devices in \VTKm is provided in Section \ref{sec:RuntimeDeviceTracker}.)

\vtkmlisting{Specifying a device using a device adapter tag.}{SpecifyDeviceAdapter.cxx}

For classes and methods that have a template argument that is expected to be a device adapter tag, the tag type can be checked with the \vtkmmacro{VTKM\_IS\_DEVICE\_ADAPTER\_TAG} macro to verify the type is a valid device adapter tag.
It is good practice to check unknown types with this macro to prevent further unexpected errors.

%% Functions, methods, and classes that directly use device adapter tags are usually templated on the device adapter tag.
%% This allows the function or class to be applied to any type of device specified at compile time.

%% \vtkmlisting[ex:DeviceTemplateArg]{Specifying a device using template parameters.}{DeviceTemplateArg.cxx}

%% \begin{commonerrors}
%%   A device adapter tag is a class just like every other type in C++.
%%   Thus it is possible to accidently use a type that is not a device adapter tag when one is expected as a template argument.
%%   This leads to unexpected errors in strange parts of the code.
%%   To help identify these errors, it is good practice to use the \vtkmmacro{VTKM\_IS\_DEVICE\_ADAPTER\_TAG} macro to verify the type is a valid device adapter tag.
%%   Example~\ref{ex:DeviceTemplateArg} uses this macro on line 4.
%% \end{commonerrors}

\index{tag!device adapter|)}
\index{device adapter!tag|)}


\section{Device Adapter Id}
\label{sec:DeviceAdapterId}

\index{device adapter!id|(}
\index{id!device adapter|(}

Using a device adapter tag directly means that the type of device needs to be known at compile time.
To store a device adapter type at run time, one can instead use \vtkmcont{DeviceAdapterId}.
\textidentifier{DeviceAdapterId} is a superclass to all the device adapter tags, and any device adapter tag can be ``stored'' in a \textidentifier{DeviceAdapterId}.
Thus, it is more common for functions and classes to use \textidentifier{DeviceAdapterId} then to try to track a specific device with templated code.

In addition to the provided device adapter tags listed previously, a \textidentifier{DeviceAdapterId} can store some special device adapter tags that do not directly specify a specific device.

\index{device adapter!id!provided|(}
\index{id!device adapter!provided|(}

\begin{description}
\item[\vtkmcont{DeviceAdapterTagAny}] \index{device adapter!any}
  Used to specify that any device may be used for an operation.
  In practice this is limited to devices that are currently available.
\item[\vtkmcont{DeviceAdapterTagUndefined}] \index{device adapter!undefined}
  Used to avoid specifying a device.
  Useful as a placeholder when a device can be specified but none is given.
\end{description}

\index{id!device adapter!provided|)}
\index{device adapter!id!provided|)}

\begin{didyouknow}
  Any device adapter tag can be used where a device adapter id is expected.
  Thus, you can use a device adapter tag whenever you want to specify a particular device and pass that to any method expecting a device id.
  Likewise, it is usually more convenient for classes and methods to manage device adapter ids rather than device adapter tag.
\end{didyouknow}

\textidentifier{DeviceAdapterId} contains several helpful methods to get runtime information about a particular device.

\begin{description}
\item[{\classmember*{DeviceAdapterId}{GetName}}]
  A \textcode{static} method that returns a string description for the device adapter.
  The string is stored in a type named \vtkmcont{DeviceAdapterNameType}, which is currently aliased to \textcode{std::string}.
  The device adapter name is useful for printing information about a device being used.
\item[{\classmember*{DeviceAdapterId}{GetId}}]
  A \textcode{static} method taking no arguments that returns a unique integer identifier for the device adapter as a \vtkm{Int8}.
\item[{\classmember*{DeviceAdapterId}{IsValueValid}}]
  A \textcode{static const bool} that is true if the implementation of the integer returned from \classmember*{DeviceAdapterId}{GetId} corresponds to a concrete device.
  So, for example, the \classmember*{DeviceAdapterId}{IsValueValid} flag for a \textidentifier{DeviceAdapterTagSerial} is true whereas the \classmember*{DeviceAdapterId}{IsValueValid} flag for a \textidentifier{DeviceAdapterTagAny} is false.
\end{description}

\begin{didyouknow}
  As a cheat, all device adapter tags actually inherit from the \vtkmcont{DeviceAdapterId} class.
  Thus, all of these methods can be called directly on a device adapter tag.
\end{didyouknow}

\begin{commonerrors}
  Just because the \classmember{DeviceAdapterId}{IsValueValid} returns true that does not necessarily mean that this device is available to be run on.
  It simply means that the device is implemented in \VTKm.
  However, that device might not be compiled, or that device might not be available on the current running system, or that device might not be enabled.
  Use the device runtime tracker described in Section \ref{sec:RuntimeDeviceTracker} to determine if a particular device can actually be used.
\end{commonerrors}

\index{id!device adapter|)}
\index{device adapter!id|)}


\section{Runtime Device Tracker}
\label{sec:RuntimeDeviceTracker}

\index{runtime device tracker|(}
\index{device adapter!runtime tracker|(}

It is often the case that you are agnostic about what device \VTKm algorithms run so long as they complete correctly and as fast as possible.
Thus, rather than directly specify a device adapter, you would like \VTKm to try using the best available device, and if that does not work try a different device.
Because of this, there are many features in \VTKm that behave this way.
For example, you may have noticed that running filters, as in the examples of Chapter \ref{chap:RunningFilters}, you do not need to specify a device; they choose a device for you.

However, even though we often would like \VTKm to choose a device for us, we still need a way to manage device preferences.
\VTKm also needs a mechanism to record runtime information about what devices are available so that it does not have to continually try (and fail) to use devices that are not available at runtime.
These needs are met with the \vtkmcont{RuntimeDeviceTracker} class.
\textidentifier{RuntimeDeviceTracker} maintains information about which devices can and should be run on.
\VTKm maintains a \textidentifier{RuntimeDeviceTracker} for each thread your code is operating on.
\index{runtime device tracker!getting}
\index{device adapter!runtime tracker!getting}
To get the runtime device for the current thread, use the \vtkmcont{GetRuntimeDeviceTracker} method.

\textidentifier{RuntimeDeviceTracker} has the following methods.

\begin{description}
\item[{\classmember*{RuntimeDeviceTracker}{CanRunOn}}]
  Takes a device adapter tag and returns true if \VTKm was configured for the device and it has not yet been marked as disabled.
\item[{\classmember*{RuntimeDeviceTracker}{DisableDevice}}]
  Takes a device adapter tag and marks that device to not be used.
  Future calls to \classmember*{RuntimeDeviceTracker}{CanRunOn} for this device will return false until that device is reset.
\item[{\classmember*{RuntimeDeviceTracker}{ResetDevice}}]
  Takes a \vtkmcont{DeviceAdapterTag} and resets the state for that device to its default value.
  Each device defaults to on as long as \VTKm is configured to use that device and a basic runtime check finds a viable device.
\item[{\classmember*{RuntimeDeviceTracker}{Reset}}]
  Resets all devices.
  This equivocally calls \classmember*{RuntimeDeviceTracker}{ResetDevice} for all devices supported by \VTKm.
\item[{\classmember*{RuntimeDeviceTracker}{ForceDevice}}]
  Takes a device adapter tag and enables that device.
  All other devices are disabled.
  This method throws a \vtkmcont{ErrorBadValue} if the requested device cannot be enabled.
%% \item[{\classmember*{RuntimeDeviceTracker}{DeepCopy}}]
%%   \textidentifier{RuntimeDeviceTracker} behaves like a smart pointer for its state.
%%   That is, if you copy a \textidentifier{RuntimeDeviceTracker} and then change the state of one (by, for example, calling \classmember*{RuntimeDeviceTracker}{DisableDevice} on one), then the state changes for both.
%%   If you want to copy the state of a \textidentifier{RuntimeDeviceTracker} but do not want future changes to effect each other, then use \classmember*{RuntimeDeviceTracker}{DeepCopy}.
%%   There are two versions of \classmember*{RuntimeDeviceTracker}{DeepCopy}.
%%   The first version takes no arguments and returns a new \textidentifier{RuntimeDeviceTracker}.
%%   The second version takes another instance of a \textidentifier{RuntimeDeviceTracker} and copies its state into this class.
\item[{\classmember*{RuntimeDeviceTracker}{ReportAllocationFailure}}]
  A device might have less working memory available than the main CPU.
  If this is the case, memory allocation errors are more likely to happen.
  This method is used to report a \vtkmcont{ErrorBadAllocation} and disables the device for future execution.
\item[{\classmember*{RuntimeDeviceTracker}{ReportBadDeviceFailure}}]
  It is possible that a device may throw a \vtkmcont{ErrorBadDevice} failure caused by some erroneous device issue.
  If this occurs, it is possible to catch the \vtkmcont{ErrorBadDevice} exception and pass it to \classmember*{RuntimeDeviceTracker}{ReportBadDeviceFailure} along with the \vtkmcont{DeviceAdapterId} to forcefully disable a device.
\end{description}

\index{runtime device tracker!scoped|(}
\index{device adapter!runtime tracker!scoped|(}
\index{scoped device adapter|(}

A \textidentifier{RuntimeDeviceTracker} can be used to specify which devices to consider for a particular operation.
However, a better way to specify devices is to use the \vtkmcont{ScopedRuntimeDeviceTracker} class.
When a \textidentifier{ScopedRuntimeDeviceTracker} is constructed, it specifies a new set of devices for \VTKm to use.
When the \textidentifier{ScopedRuntimeDeviceTracker} is destroyed as it leaves scope, it restores \VTKm's devices to those that existed when it was created.

The following example demonstrates how the \textidentifier{ScopedRuntimeDeviceTracker} is used to force the \VTKm operations that happen within a function to operate exclusively with the TBB device.

\vtkmlisting[ex:ForceThreadLocalDevice]{Restricting which devices \VTKm uses per thread.}{ForceThreadLocalDevice.cxx}

In the previous example we forced \VTKm to use the TBB device.
This is the default behavior of \textidentifier{ScopedRuntimeDeviceTracker}, but the constructor takes an optional second argument that is a value in the \vtkmcont{RuntimeDeviceTrackerMode} to specify how modify the current device adapter list.

\begin{description}
\item[\classmember{RuntimeDeviceTrackerMode}{Force}]
  Replaces the current list of devices to try with the device specified to the \textidentifier{ScopedRuntimeDeviceTracker}.
  This has the effect of forcing \VTKm to use the provided device.
  This is the default behavior for the \textidentifier{ScopedRuntimeDeviceTracker}.
\item[\classmember{RuntimeDeviceTrackerMode}{Enable}]
  Adds the provided device adapter to the list of devices to try.
\item[\classmember{RuntimeDeviceTrackerMode}{Disable}]
  Removes the provided device adapter from the list of devices to try.
\end{description}

As a motivating example, let us say that we want to perform a deep copy of an array (described in Section~\ref{sec:DeepArrayCopies}).
However, we do not want to do the copy on a CUDA device because we happen to know the data is not on that device and we do not want to spend the time to transfer the data to that device.
We can use a \vtkmcont{ScopedRuntimeDeviceTracker} to temporarily disable the CUDA device for this operation.

\vtkmlisting[ex:RestrictCopyDevice]{Disabling a device with \textidentifier{RuntimeDeviceTracker}.}{RestrictCopyDevice.cxx}

\index{scoped device adapter|)}
\index{device adapter!runtime tracker!scoped|)}
\index{runtime device tracker!scoped|)}

\index{device adapter!runtime tracker|)}
\index{runtime device tracker|)}
