Commit b0303c96 authored by Kenneth Moreland's avatar Kenneth Moreland
Browse files

Allow Initialize to parse only some arguments

When a library requires reading some command line arguments through a
function like Initialize, it is typical that it will parse through
arguments it supports and then remove those arguments from argc and argv
so that the remaining arguments can be parsed by the calling program.
VTK-m's initialize did not do that, so add that functionality.
parent d1a957bf
# Allow Initialize to parse only some arguments
When a library requires reading some command line arguments through a
function like Initialize, it is typical that it will parse through
arguments it supports and then remove those arguments from `argc` and
`argv` so that the remaining arguments can be parsed by the calling
program. Recent changes to the `vtkm::cont::Initialize` function support
that.
## Use Case
Say you are creating a simple benchmark where you want to provide a command
line option `--size` that allows you to adjust the size of the data that
you are working on. However, you also want to support flags like `--device`
and `-v` that are performed by `vtkm::cont::Initialize`. Rather than have
to re-implement all of `Initialize`'s parsing, you can now first call
`Initialize` to handle its arguments and then parse the remaining objects.
The following is a simple (and rather incomplete) example:
```cpp
int main(int argc, char** argv)
{
vtkm::cont::InitializeResult initResult = vtkm::cont::Initialize(argc, argv);
if ((argc > 1) && (strcmp(argv[1], "--size") == 0))
{
if (argc < 3)
{
std::cerr << "--size option requires a numeric argument" << std::endl;
std::cerr << "USAGE: " << argv[0] << " [options]" << std::endl;
std::cerr << "Options are:" << std::endl;
std::cerr << " --size <number>\tSpecify the size of the data." << std::endl;
std::cerr << initResult.Usage << std::endl;
exit(1);
}
g_size = atoi(argv[2]);
}
std::cout << "Using device: " << initResult.Device.GetName() << std::endl;
```
## Additional Initialize Options
Because `Initialize` no longer has the assumption that it is responsible
for parsing _all_ arguments, some options have been added to
`vtkm::cont::InitializeOptions` to manage these different use cases. The
following options are now supported.
* `None` A placeholder for having all options off, which is the default.
(Same as before this change.)
* `RequireDevice` Issue an error if the device argument is not specified.
(Same as before this change.)
* `DefaultAnyDevice` If no device is specified, treat it as if the user
gave --device=Any. This means that DeviceAdapterTagUndefined will never
be return in the result.
* `AddHelp` Add a help argument. If `-h` or `--help` is provided, prints
a usage statement. Of course, the usage statement will only print out
arguments processed by VTK-m.
* `ErrorOnBadOption` If an unknown option is encountered, the program
terminates with an error and a usage statement is printed. If this
option is not provided, any unknown options are returned in `argv`. If
this option is used, it is a good idea to use `AddHelp` as well.
* `ErrorOnBadArgument` If an extra argument is encountered, the program
terminates with an error and a usage statement is printed. If this
option is not provided, any unknown arguments are returned in `argv`.
* `Strict` If supplied, Initialize treats its own arguments as the only
ones supported by the application and provides an error if not followed
exactly. This is a convenience option that is a combination of
`ErrorOnBadOption`, `ErrorOnBadArgument`, and `AddHelp`.
## InitializeResult Changes
The changes in `Initialize` have also necessitated the changing of some of
the fields in the `InitializeResult` structure. The following fields are
now provided in the `InitializeResult` struct.
* `Device` Returns the device selected in the command line arguments as a
`DeviceAdapterId`. If no device was selected,
`DeviceAdapterTagUndefined` is returned. (Same as before this change.)
* `Usage` Returns a string containing the usage for the options
recognized by `Initialize`. This can be used to build larger usage
statements containing options for both `Initialize` and the calling
program. See the example above.
Note that the `Arguments` field has been removed from `InitializeResult`.
This is because the unparsed arguments are now returned in the modified
`argc` and `argv`, which provides a more complete result than the
`Arguments` field did.
......@@ -34,7 +34,7 @@ namespace
enum OptionIndex
{
PREAMBLE, // usage header strings
UNKNOWN,
DEVICE,
LOGLEVEL, // not parsed by this parser, but by loguru
HELP
......@@ -118,19 +118,56 @@ struct VtkmArg : public opt::Arg
}
return names.str();
}
};
static opt::Descriptor Usage[] = {
{ PREAMBLE, 0, "", "", opt::Arg::None, "Usage information:\n" },
{ DEVICE,
0,
"d",
"device",
VtkmArg::IsDevice,
" --device, -d [dev] \tForce device to dev. Omit device to list available devices." },
{ LOGLEVEL, 0, "v", "", opt::Arg::None, " -v \tSpecify a log level (when logging is enabled)." },
{ HELP, 0, "h", "help", opt::Arg::None, " --help, -h \tPrint usage information." },
{ 0, 0, 0, 0, 0, 0 }
static opt::ArgStatus Required(const opt::Option& option, bool msg)
{
if (option.arg == nullptr)
{
if (msg)
{
VTKM_LOG_ALWAYS_S(vtkm::cont::LogLevel::Error,
"Missing argument after option '"
<< std::string(option.name, static_cast<size_t>(option.namelen))
<< "'.\n");
}
return opt::ARG_ILLEGAL;
}
else
{
return opt::ARG_OK;
}
}
// Method used for guessing whether an option that do not support (perhaps that calling
// program knows about it) has an option attached to it (which should also be ignored).
static opt::ArgStatus UnknownOption(const opt::Option& option, bool msg)
{
// If we don't have an arg, obviously we don't have an arg.
if (option.arg == nullptr)
{
return opt::ARG_NONE;
}
// The opt::Arg::Optional method will return that the ARG is OK if and only if
// the argument is attached to the option (e.g. --foo=bar). If that is the case,
// then we definitely want to report that the argument is OK.
if (opt::Arg::Optional(option, msg) == opt::ARG_OK)
{
return opt::ARG_OK;
}
// Now things get tricky. Maybe the next argument is an option or maybe it is an
// argument for this option. We will guess that if the next argument does not
// look like an option, we will treat it as such.
if (option.arg[0] == '-')
{
return opt::ARG_NONE;
}
else
{
return opt::ARG_OK;
}
}
};
} // end anon namespace
......@@ -155,51 +192,171 @@ InitializeResult Initialize(int& argc, char* argv[], InitializeOptions opts)
vtkm::cont::InitLogging(argc, argv);
}
{ // Parse VTKm options:
{ // Parse VTKm options
std::vector<opt::Descriptor> usage;
if ((opts & InitializeOptions::AddHelp) != InitializeOptions::None)
{
usage.push_back({ UNKNOWN, 0, "", "", VtkmArg::UnknownOption, "Usage information:\n" });
}
usage.push_back(
{ DEVICE,
0,
"d",
"device",
VtkmArg::IsDevice,
" --device, -d <dev> \tForce device to dev. Omit device to list available devices." });
usage.push_back(
{ LOGLEVEL,
0,
"v",
"",
VtkmArg::Required,
" -v <#|INFO|WARNING|ERROR|FATAL|OFF> \tSpecify a log level (when logging is enabled)." });
if ((opts & InitializeOptions::AddHelp) != InitializeOptions::None)
{
usage.push_back(
{ HELP, 0, "h", "help", opt::Arg::None, " --help, -h \tPrint usage information." });
}
// Required to collect unknown arguments when help is off.
usage.push_back({ UNKNOWN, 0, "", "", VtkmArg::UnknownOption, "" });
usage.push_back({ 0, 0, 0, 0, 0, 0 });
{
std::stringstream streamBuffer;
opt::printUsage(streamBuffer, usage.data());
config.Usage = streamBuffer.str();
// Remove trailing newline as one more than we want is added.
config.Usage = config.Usage.substr(0, config.Usage.length() - 1);
}
// Remove argv[0] (executable name) if present:
int vtkmArgc = argc > 0 ? argc - 1 : 0;
char** vtkmArgv = vtkmArgc > 0 ? argv + 1 : argv;
opt::Stats stats(Usage, argc, argv);
opt::Stats stats(usage.data(), vtkmArgc, vtkmArgv);
std::unique_ptr<opt::Option[]> options{ new opt::Option[stats.options_max] };
std::unique_ptr<opt::Option[]> buffer{ new opt::Option[stats.buffer_max] };
opt::Parser parse(Usage, vtkmArgc, vtkmArgv, options.get(), buffer.get());
opt::Parser parse(usage.data(), vtkmArgc, vtkmArgv, options.get(), buffer.get());
if (parse.error())
{
opt::printUsage(std::cerr, Usage);
std::cerr << config.Usage;
exit(1);
}
if (options[HELP])
{
opt::printUsage(std::cout, Usage);
std::cerr << config.Usage;
exit(0);
}
if (options[DEVICE])
{
auto id = vtkm::cont::make_DeviceAdapterId(options[DEVICE].arg);
auto& tracker = vtkm::cont::GetRuntimeDeviceTracker();
tracker.ForceDevice(id);
if (id != vtkm::cont::DeviceAdapterTagAny{})
{
vtkm::cont::GetRuntimeDeviceTracker().ForceDevice(id);
}
else
{
vtkm::cont::GetRuntimeDeviceTracker().Reset();
}
config.Device = id;
}
else if ((opts & InitializeOptions::DefaultAnyDevice) != InitializeOptions::None)
{
vtkm::cont::GetRuntimeDeviceTracker().Reset();
config.Device = vtkm::cont::DeviceAdapterTagAny{};
}
else if ((opts & InitializeOptions::RequireDevice) != InitializeOptions::None)
{
auto devices = VtkmArg::GetValidDeviceNames();
VTKM_LOG_ALWAYS_S(vtkm::cont::LogLevel::Error,
"Target device must be specified via -d or --device.\n"
"Valid devices: "
<< devices
<< "\n");
opt::printUsage(std::cerr, Usage);
VTKM_LOG_S(vtkm::cont::LogLevel::Error, "Device not given on command line.");
std::cerr << "Target device must be specified via -d or --device.\n"
"Valid devices: "
<< devices << std::endl;
if ((opts & InitializeOptions::AddHelp) != InitializeOptions::None)
{
std::cerr << config.Usage;
}
exit(1);
}
for (int i = 0; i < parse.nonOptionsCount(); i++)
for (const opt::Option* opt = options[UNKNOWN]; opt != nullptr; opt = opt->next())
{
VTKM_LOG_S(vtkm::cont::LogLevel::Info, "Unknown option to Initialize: " << opt->name << "\n");
if ((opts & InitializeOptions::ErrorOnBadOption) != InitializeOptions::None)
{
std::cerr << "Unknown option: " << opt->name << std::endl;
if ((opts & InitializeOptions::AddHelp) != InitializeOptions::None)
{
std::cerr << config.Usage;
}
exit(1);
}
}
for (int nonOpt = 0; nonOpt < parse.nonOptionsCount(); ++nonOpt)
{
VTKM_LOG_S(vtkm::cont::LogLevel::Info,
"Unknown argument to Initialize: " << parse.nonOption(nonOpt) << "\n");
if ((opts & InitializeOptions::ErrorOnBadArgument) != InitializeOptions::None)
{
std::cerr << "Unknown argument: " << parse.nonOption(nonOpt) << std::endl;
if ((opts & InitializeOptions::AddHelp) != InitializeOptions::None)
{
std::cerr << config.Usage;
}
exit(1);
}
}
// Now go back through the arg list and remove anything that is not in the list of
// unknown options or non-option arguments.
int destArg = 1;
for (int srcArg = 1; srcArg < argc; ++srcArg)
{
config.Arguments.emplace_back(parse.nonOption(i));
std::string thisArg{ argv[srcArg] };
bool copyArg = false;
// Special case: "--" gets removed by optionparser but should be passed.
if (thisArg == "--")
{
copyArg = true;
}
for (const opt::Option* opt = options[UNKNOWN]; !copyArg && opt != nullptr; opt = opt->next())
{
if (thisArg == opt->name)
{
copyArg = true;
}
if ((opt->arg != nullptr) && (thisArg == opt->arg))
{
copyArg = true;
}
// Special case: optionparser sometimes removes a single "-" from an option
if (thisArg.substr(1) == opt->name)
{
copyArg = true;
}
}
for (int nonOpt = 0; !copyArg && nonOpt < parse.nonOptionsCount(); ++nonOpt)
{
if (thisArg == parse.nonOption(nonOpt))
{
copyArg = true;
}
}
if (copyArg)
{
if (destArg != srcArg)
{
argv[destArg] = argv[srcArg];
}
++destArg;
}
}
argc = destArg;
}
return config;
......
......@@ -38,14 +38,39 @@ struct InitializeResult
/// Device passed into -d, or undefined
DeviceAdapterId Device = DeviceAdapterTagUndefined{};
/// Non-option arguments
std::vector<std::string> Arguments;
/// Usage statement for arguments parsed by VTK-m
std::string Usage;
};
enum class InitializeOptions
{
None = 0x0,
RequireDevice = 0x1
None = 0x00,
/// Issue an error if the device argument is not specified.
RequireDevice = 0x01,
/// If no device is specified, treat it as if the user gave --device=Any. This means that
/// DeviceAdapterTagUndefined will never be return in the result.
DefaultAnyDevice = 0x02,
/// Add a help argument. If -h or --help is provided, prints a usage statement. Of course,
/// the usage statement will only print out arguments processed by VTK-m.
AddHelp = 0x04,
/// If an unknown option is encountered, the program terminates with an error and a usage
/// statement is printed. If this option is not provided, any unknown options are returned
/// in argv. If this option is used, it is a good idea to use AddHelp as well.
ErrorOnBadOption = 0x08,
/// If an extra argument is encountered, the program terminates with an error and a usage
/// statement is printed. If this option is not provided, any unknown arguments are returned
/// in argv.
ErrorOnBadArgument = 0x10,
/// If supplied, Initialize treats its own arguments as the only ones supported by the
/// application and provides an error if not followed exactly. This is a convenience
/// option that is a combination of ErrorOnBadOption, ErrorOnBadArgument, and AddHelp.
Strict = ErrorOnBadOption | ErrorOnBadArgument | AddHelp
};
// Allow options to be used as a bitfield
......
......@@ -75,6 +75,7 @@ set(unit_tests
UnitTestDeviceAdapterAlgorithmGeneral.cxx
UnitTestDynamicCellSet.cxx
UnitTestFieldRangeCompute.cxx
UnitTestInitialize.cxx
UnitTestLogging.cxx
UnitTestMoveConstructors.cxx
UnitTestMultiBlock.cxx
......
//============================================================================
// Copyright (c) Kitware, Inc.
// All rights reserved.
// See LICENSE.txt for details.
// This software is distributed WITHOUT ANY WARRANTY; without even
// the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
// PURPOSE. See the above copyright notice for more information.
//
// Copyright 2019 National Technology & Engineering Solutions of Sandia, LLC (NTESS).
// Copyright 2019 UT-Battelle, LLC.
// Copyright 2019 Los Alamos National Security.
//
// Under the terms of Contract DE-NA0003525 with NTESS,
// the U.S. Government retains certain rights in this software.
//
// Under the terms of Contract DE-AC52-06NA25396 with Los Alamos National
// Laboratory (LANL), the U.S. Government retains certain rights in
// this software.
//============================================================================
#include <vtkm/cont/Initialize.h>
#include <vtkm/cont/testing/Testing.h>
namespace
{
constexpr const char* PROGRAM_NAME = "program-name";
template <typename... T>
void MakeArgs(int& argc, char**& argv, T&&... args)
{
constexpr std::size_t numArgs = sizeof...(args) + 1;
std::array<std::string, numArgs> stringArgs = { { PROGRAM_NAME, args... } };
// These static variables are declared as static so that the memory will stick around but won't
// be reported as a leak.
static std::array<std::vector<char>, numArgs> vecArgs;
static std::array<char*, numArgs + 1> finalArgs;
std::cout << " starting args:";
for (std::size_t i = 0; i < numArgs; ++i)
{
std::cout << " " << stringArgs[i];
// Safely copying a C-style string is a PITA
vecArgs[i].resize(0);
vecArgs[i].reserve(stringArgs[i].size() + 1);
for (auto&& c : stringArgs[i])
{
vecArgs[i].push_back(c);
}
vecArgs[i].push_back('\0');
finalArgs[i] = vecArgs[i].data();
}
finalArgs[numArgs] = nullptr;
std::cout << std::endl;
argc = static_cast<int>(numArgs);
argv = finalArgs.data();
}
template <typename... T>
void CheckArgs(int argc, char* argv[], T&&... args)
{
constexpr std::size_t numArgs = sizeof...(args) + 1;
std::array<std::string, numArgs> expectedArgs = { { PROGRAM_NAME, args... } };
std::cout << " expected args:";
for (std::size_t i = 0; i < numArgs; ++i)
{
std::cout << " " << expectedArgs[i];
}
std::cout << std::endl;
std::cout << " received args:";
for (int i = 0; i < argc; ++i)
{
std::cout << " " << argv[i];
}
std::cout << std::endl;
VTKM_TEST_ASSERT(
numArgs == static_cast<std::size_t>(argc), "Got wrong number of arguments (", argc, ")");
for (std::size_t i = 0; i < numArgs; ++i)
{
VTKM_TEST_ASSERT(expectedArgs[i] == argv[i], "Arg ", i, " wrong");
}
std::cout << std::endl;
}
void InitializeZeroArguments()
{
std::cout << "Initialize with no arguments" << std::endl;
vtkm::cont::Initialize();
}
void InitializeNoOptions()
{
std::cout << "Initialize without any options" << std::endl;
int argc;
char** argv;
MakeArgs(argc, argv);
vtkm::cont::InitializeResult result = vtkm::cont::Initialize(argc, argv);
CheckArgs(argc, argv);
std::cout << "Usage statement returned from Initialize:" << std::endl;
std::cout << result.Usage << std::endl;
}
void InitializeStandardOptions()
{
std::cout << "Initialize with some standard options" << std::endl;
int argc;
char** argv;
MakeArgs(argc, argv, "--device", "Any");
vtkm::cont::Initialize(argc, argv, vtkm::cont::InitializeOptions::Strict);
CheckArgs(argc, argv);
}
void InitializeCustomOptions()
{
std::cout << "Initialize with some custom options and arguments" << std::endl;
int argc;
char** argv;
MakeArgs(argc, argv, "--foo", "-bar", "baz", "buz");
vtkm::cont::Initialize(argc, argv);
CheckArgs(argc, argv, "--foo", "-bar", "baz", "buz");
MakeArgs(argc, argv, "--foo", "-bar", "--", "baz", "buz");
vtkm::cont::Initialize(argc, argv);
CheckArgs(argc, argv, "--foo", "-bar", "--", "baz", "buz");
}
void InitializeMixedOptions()
{
std::cout << "Initialize with options both for VTK-m and some that are not." << std::endl;
int argc;
char** argv;
MakeArgs(argc, argv, "--foo", "--device", "Any", "--bar", "baz");
vtkm::cont::Initialize(argc, argv, vtkm::cont::InitializeOptions::AddHelp);
CheckArgs(argc, argv, "--foo", "--bar", "baz");
MakeArgs(argc, argv, "--foo", "-v", "OFF", "--", "--device", "Any", "--bar", "baz");
vtkm::cont::Initialize(argc, argv);
CheckArgs(argc, argv, "--foo", "--", "--device", "Any", "--bar", "baz");
MakeArgs(argc, argv, "--device", "Any", "foo");
vtkm::cont::Initialize(argc, argv);
CheckArgs(argc, argv, "foo");
}
void InitializeCustomOptionsWithArgs()
{
std::cout << "Calling program has option --foo that takes arg bar." << std::endl;
int argc;
char** argv;
MakeArgs(argc, argv, "--device", "Any", "--foo=bar", "--baz");
vtkm::cont::Initialize(argc, argv);
CheckArgs(argc, argv, "--foo=bar", "--baz");
MakeArgs(argc, argv, "--foo=bar", "--baz", "--device", "Any");
vtkm::cont::Initialize(argc, argv);
CheckArgs(argc, argv, "--foo=bar", "--baz");
MakeArgs(argc, argv, "--device", "Any", "--foo", "bar", "--baz");
vtkm::cont::Initialize(argc, argv);
CheckArgs(argc, argv, "--foo", "bar", "--baz");
MakeArgs(argc, argv, "--foo", "bar", "--baz", "--device", "Any");
vtkm::cont::Initialize(argc, argv);
CheckArgs(argc, argv, "--foo", "bar", "--baz");
}
void InitializeWithHelp()
{
std::cout << "Pass help flag to initialize" << std::endl;
int argc;
char** argv;
MakeArgs(argc, argv, "--help");
vtkm::cont::Initialize(argc, argv, vtkm::cont::InitializeOptions::AddHelp);
VTKM_TEST_FAIL("Help argument did not exit as expected.");
}
void DoInitializeTests()