Implementing nuget package restore.
This issue regards the design questions raised in !6761 (merged).
Overview
When generating msbuild projects, CMake has the possibility to define VS_PACKAGE_REFERENCE
s. Those are used to define dependencies for NuGet packages in Visual Studio projects. NuGet is a package manager for both, managed (.NET) and unmanaged (native) projects. NuGet packages are provided on a remote package store, that can either be the central NuGet gallery, or a custom source.
If a project defines a VS_PACKAGE_REFERENCE
, it defines a mandatory dependency. Unmanaged targets may require link libraries from the package, whilst managed targets may face runtime errors, if a required assembly from the target does not exist. In order to make sure, the dependent packages exist, they need to be restored. Restoring packages in this context means, that the build process makes sure, the packages are available (i.e. downloaded to the local package cache, which is nothing more than a folder on the local system) and writing a nuget asset file, that is used by the build tool (msbuild) to unpack and copy over the assemblies or link libraries to the build directory. If a package dependency does not exist within the local package cache, it will be installed. Installing involves looking up the package from the defined remote sources and downloading it to the local package cache.
Problem Description
When defining a VS_PACKAGE_REFERENCE
in the current version of CMake, one has to make sure that the package is installed before building the project. If the CMake-generated solution is built from within Visual Studio, a restore will automatically be executed by Visual Studio (if not configured otherwise). If, however, the project is built using CMake from command line, e.g. by:
cmake --build . -config Release
The build will fail, if the referenced packages have not been manually restored earlier. This is especially problematic when using the ninja generator (for example when using the CMake-integration with Visual Studio). One could run a restore on the targets before the build, however, since NuGet packages may alter MSBuild targets, it is not guaranteed to work. A workaround would be to pass the restore flag to msbuild manually:
cmake --build . -config Release -- -r
This does only work for msbuild 15.5 and above. Earlier versions do not yet support the restore flag and require a separate Restore
target to be built. However, this target must not be build build alongside the actual project, but instead must be invoked before the actual build. Note that versions before msbuild 15.0 do not support any form of restore.
Proposed Solution
Issue #20646 (closed) discussed automatic package restore to be integrated into CMake before. The idea was to run restore at generation time. This is problematic, since the package cache might be altered between generation time and build time.
!6761 (merged) thus currently implements restore during build time. It does this, depending on the msbuild version:
- For msbuild 15.5 and above, it simply adds the
-restore
flag to the make command. This is trivial, since this flag does not have any effect, if no package references are defined. - For earlier msbuild versions until 15.0, it adds an additional make command to invoke
-t:Restore
on the targets that should be built. It does this only, if the target itself or any direct dependency have package references defined, so if NuGet is not used, no additional make command is generated.
During review of the MR, the issue has been raised, that restore might require a network connection. This is only true, if the referenced packages have not been earlier installed. If the package cache is up-to-date with the package references of the target, no network connection is required. So if the build process must be run in such a protected environment, it is possible to first install all packages from any remote source in a pre-build process (for example by using nuget install <package>
), then disconnect from the network and resume the build process with a normal restore
. Restoring in this case will only write the nuget asset file to the intermediate directory. Thus the proposed implementation would also work in such a scenario without further modifications.
Design Decisions
Visual Studio allows for automatic package restore to be disabled. #20646 (closed) proposed to introduce a cache variable CMAKE_VS_NUGET_PACKAGE_RESTORE
as an equivalent way to disable package restore. !6761 (merged) does implement such a cache variable and sets it to ON
by default. The reasoning for this is:
- A cache variable lets package restore be toggled on/off on a per-user-level. It has been proposed to make this variable a command line parameter (e.g.
--skip-nuget-package-restore
). This would make this switch less "granular", since it would need to be passed to the command line during build (for example in a preset or build script). - Defaulting to
ON
resembles the default behavior of Visual Studio. Package references are mandatory for reasons stated above. There might be situations where one wants to disable automatic restore, but the general advise is to not disable it, since the build may fail or reference the wrong package versions. Furthermore, restore is mandatory for the newly "SDK-style" projects, that are implemented by !6634 (merged).
In order to continue implementing !6761 (merged), we need to decide on those two questions. Cache variable or command line parameter? Default restore to ON
/OFF
? If more questions arise, we can also discuss them here.