Symlink implementation changes in 3.13 may be causing packaging regressions (#18726) · Issues · CMake / CMake

Symlink implementation changes in 3.13 may be causing packaging regressions

Setup

CMake 3.12.3 & 3.13.2
macOS 10.14.2
Xcode 10.1
CPACK_GENERATOR "DragNDrop"
https://github.com/openMW/openmw/

Issue

I started observing the issue as I upgraded CMake to 3.13.2. As a part of macOS app bundle fixup, CPack copies all dependencies (libraries, frameworks) to the bundle. Some of the dependencies contain not only binaries but also symlinks, for example there's libosg.3.4.1.dylib and a symlink to it called libosg.130.dylib. Binaries in the bundle are linked against the symlink name.

Here's a part of the verbose CPack output with CMake 3.12.3:

...
CPack Verbose: fixup_bundle: copying...
...
CPack Verbose: 3/94: linking '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/MacOS/libOpenThreads.3.3.0.dylib' -> '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/MacOS/libOpenThreads.20.dylib'
CPack Verbose: 4/94: linking '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/MacOS/libosg.3.4.1.dylib' -> '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/MacOS/libosg.130.dylib'
CPack Verbose: 5/94: linking '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/MacOS/libosgDB.3.4.1.dylib' -> '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/MacOS/libosgDB.130.dylib'
CPack Verbose: 6/94: linking '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/MacOS/libosgUtil.3.4.1.dylib' -> '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/MacOS/libosgUtil.130.dylib'
...
CPack Verbose: 39/94: copying '/Users/username/Projects/openmw/openmw-deps/build-55c27b1dd68517215acb3ed36d1991d2aef465fe/openmw-deps/lib/libOpenThreads.3.3.0.dylib'
CPack Verbose: 40/94: copying '/Users/username/Projects/openmw/openmw-deps/build-55c27b1dd68517215acb3ed36d1991d2aef465fe/openmw-deps/lib/libosg.3.4.1.dylib'
CPack Verbose: 41/94: copying '/Users/username/Projects/openmw/openmw-deps/build-55c27b1dd68517215acb3ed36d1991d2aef465fe/openmw-deps/lib/libosgDB.3.4.1.dylib'
CPack Verbose: 42/94: copying '/Users/username/Projects/openmw/openmw-deps/build-55c27b1dd68517215acb3ed36d1991d2aef465fe/openmw-deps/lib/libosgUtil.3.4.1.dylib'
...

Please note that symlinks are "copied" into the bundle before the actual files are copied, thus making the symlinks dangling until after the steps 39-42.

Here's what's happening on CMake 3.13.2:

...
CPack Verbose: 3/94: linking '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libOpenThreads.3.3.0.dylib' -> '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libOpenThreads.20.dylib'
CMake Error: failed to create symbolic link '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libOpenThreads.20.dylib': no such file or directory
CPack Verbose: 4/94: linking '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosg.3.4.1.dylib' -> '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosg.130.dylib'
CMake Error: failed to create symbolic link '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosg.130.dylib': no such file or directory
CPack Verbose: 5/94: linking '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosgDB.3.4.1.dylib' -> '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosgDB.130.dylib'
CMake Error: failed to create symbolic link '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosgDB.130.dylib': no such file or directory
CPack Verbose: 6/94: linking '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosgUtil.3.4.1.dylib' -> '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosgUtil.130.dylib'
CMake Error: failed to create symbolic link '/Users/username/Projects/openmw/openmw-build-release/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosgUtil.130.dylib': no such file or directory
...

As you can see, CMake 3.13.2 refuses to create dangling symlinks, thus breaking the bundle fixup process.

I believe that this symlink implementation change should be either announced as a breaking change with some hints on how to fix packaging process or fixed.

I can only guess but this looks like a potential culprit: a9df54ec.

Steps to reproduce

Clone https://github.com/OpenMW/openmw.git
Download & unpack dependencies: https://github.com/OpenMW/openmw.git
Download QT. A version from Homebrew should suffice: brew install qt
Create a build directory & run CMake:

cmake \
-D CMAKE_PREFIX_PATH="/path/to/unpacked/dependencies;`brew --prefix qt`" \
-D CMAKE_OSX_DEPLOYMENT_TARGET="10.9" \
-D CMAKE_OSX_SYSROOT="macosx10.14" \
-D CMAKE_BUILD_TYPE=Release \
-D OPENMW_OSX_DEPLOYMENT=TRUE \
-D DESIRED_QT_VERSION=5 \
-D BUILD_ESMTOOL=FALSE \
-D BUILD_MYGUI_PLUGIN=FALSE \
-G"Unix Makefiles" \
path/to/openmw-checkout

make -j4 package

Expected results

No packaging errors.

Actual results:

CMake Error: failed to create symbolic link '/Users/travis/build/OpenMW/openmw/build/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libOpenThreads.20.dylib': no such file or directory
CMake Error: failed to create symbolic link '/Users/travis/build/OpenMW/openmw/build/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosg.130.dylib': no such file or directory
CMake Error: failed to create symbolic link '/Users/travis/build/OpenMW/openmw/build/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosgDB.130.dylib': no such file or directory
CMake Error: failed to create symbolic link '/Users/travis/build/OpenMW/openmw/build/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW.app/Contents/Frameworks/libosgUtil.130.dylib': no such file or directory
CMake Error: failed to create symbolic link '/Users/travis/build/OpenMW/openmw/build/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW-CS.app/Contents/Frameworks/libOpenThreads.20.dylib': no such file or directory
CMake Error: failed to create symbolic link '/Users/travis/build/OpenMW/openmw/build/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW-CS.app/Contents/Frameworks/libosg.130.dylib': no such file or directory
CMake Error: failed to create symbolic link '/Users/travis/build/OpenMW/openmw/build/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW-CS.app/Contents/Frameworks/libosgDB.130.dylib': no such file or directory
CMake Error: failed to create symbolic link '/Users/travis/build/OpenMW/openmw/build/_CPack_Packages/Darwin/DragNDrop/OpenMW-0.46.0-Darwin/ALL_IN_ONE/OpenMW-CS.app/Contents/Frameworks/libosgUtil.130.dylib': no such file or directory

Edited Dec 25, 2018 by Nikolay Kasyanov

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information