Commit fc0ed484 authored by Gabriel Devillers's avatar Gabriel Devillers
Browse files

Developer Guide: adapt to superbuild update

parent ba9de095
# Table of contents
1. [VeloView dependencies](#dependencies)
1. [PCAP library](#pcap-library)
2. [Boost library](#boost-library)
3. [Qt library](#qt-library)
4. [Python](#python)
5. [Python Qt library](#python-qt-library)
6. [Paraview and VTK](#paraview-vtk)
2. [Configure and build instructions](#configure-build)
1. [Superbuild Overview](#superbuild-overview)
2. [Windows dependencies](#windows-dependencies)
3. [Windows build instructions](#windows-build-instructions)
2. [Linux dependencies](#linux-dependencies)
3. [Linux build instructions](#linux-build-instructions)
## VeloView dependencies <a name="dependencies"></a>
The VeloView application and libraries have several external library dependencies. As explained in [Superbuild Overview](#superbuild-overview), **most of the dependencies will be downloaded and compiled automatically** during the build step. See [Configure and build instructions](#configure-build).
### PCAP library <a name="pcap-library"></a>
The required pcap version is 1.4.
Pcap is required to support saving captured packets to a file, and reading files containing saved packets.
On Mac/Linux, only libpcap is required.
On Windows, we use the Winpcap project which includes libpcap but also includes Windows specific drivers. Since the winpcap project only provides Visual Studio project files, which may be out dated, the superbuild does not attempt to compile winpcap. Instead, a git repository containing headers and precompiled .lib and .dll files is used. The repository url is https://github.com/patmarion/winpcap.
### Boost library <a name="boost-library"></a>
The required boost version is 1.63.
Boost is used for threading and synchronization, network communication and handling of the filesystem.
### Qt library <a name="qt-library"></a>
The required Qt version is 5.10.
Qt is a desktop widget library that is used to provide user interface elements like windows and menus across the supported platforms Windows, Mac, and Linux.
### Python <a name="python"></a>
The required Python version is 2.7.
VeloView uses libpython to embed a Python interpreter in the VeloView application.
The core VeloView features are implemented in C++ libraries, and the libraries are wrapped for Python using VTK's Python wrapping tools.
### PythonQt <a name="python-qt-library"></a>
PythonQt version is "patch_8" (see Superbuild/version.txt).
PythonQt is used to build Qt applications using Python.
PythonQt has support for wrapping types derived from Qt objects and VTK objects.
### Paraview (and VTK) <a name="paraview-vtk"></a>
The required ParaView version is 5.4.
The required VTK version is 8.1.
The ParaView repository includes VTK, so the superbuild only needs to checkout and build ParaView in order to satisfy both dependencies.
A specific git commit sha1 is used instead of a specific released version.
The commit sha1 is very similar to the version 4.0 release but it has a few commits from the ParaView master branch cherry-picked onto it.
The commits added are those that resolve some issues with the Python console and add the PythonQtPlugin for ParaView.
The PythonQtPlugin is a small plugin that initializes the PythonQt library and makes it available in the ParaView Python console.
## Configure and build instructions <a name="configure-build"></a>
The VeloView software is hosted in git repositories that live on github.com (public version) and gitlab.kitware.com (internal version).
### Superbuild Overview <a name="superbuild-overview"></a>
VeloView can use a cmake *superbuild* to download and compile third party projects that are dependencies of VeloView.
The superbuild is not mandatory but it is recommended. It eases building a lot for new developers.
The superbuild will give you the option to use system installations of third party projects instead of compiling them as a superbuild step.
Some dependencies, on certain platforms, must be compiled by the superbuild, and for them there is no option to use a system version.
### Windows dependencies <a name="windows-dependencies"></a>
- cmake version 3.7.2 is confirmed to work (lower versions may not work, higher versions will work), cmake is available at <https://cmake.org/>
- ninja version 1.8.2 or higher, available at <https://github.com/ninja-build/ninja/releases>. There is no installer for this tool. You must extract the binary ninja.exe from ninja-win.zip and place it inside a directory that is inside your %PATH% environnement variable, such as C:\\Windows
- Microsoft Visual Studio **14** (2015) **Express** ("Desktop"). You can use this link to download the installer: <http://go.microsoft.com/fwlink/?LinkId=615464> This installer is pretty simple (no special options).
- git: we recommand using "Git for Windows" available at <https://gitforwindows.org/>
- Qt 5.10.0 *(this dependency will be built automatically in the future)*. You can download the installer here: <https://download.qt.io/official_releases/qt/5.10/5.10.0/qt-opensource-windows-x86-5.10.0.exe>. When installing you can keep the suggested installation path. Here is a walkthrough of the installer: click "Next" > "Skip" > "Next" > keep default install path (advised) and click "Next" > Unfold "Qt" then unfold "Qt 5.10.0" and tick "**MSVC 2015 64-bits**" then click "Next" > "Next" > "Install" > wait for it to install then click "Next" > untick "Launch Qt Creator" and click "Finish"
### Windows build instructions <a name="windows-build-instructions"></a>
1. clone VeloView's source code repository to a directory of your chosing, for example:
`cd <work-directory>`
`git clone <git url to VeloView repository> VeloView-source`
* you will have to know the path to this directory for step 6.
* moving this directoy in the future will break all build environnements that were using it (you will have to redo steps 6. and 7.)
2. clone the submodule that is used to provide the Superbuild:
`cd VeloView-source`
`git submodule update --init Superbuild/common-superbuild`
* this command must be executed inside the previously cloned repository
3. create a new directory to store the build.
`mkdir C:\VeloView-build`
* You can use the Windows file explorer to create this directory
* **This directory must not be inside the VeloView source code directory**
* **the path to this directory must be short** because Windows has limitations on the maximum length of file paths. We suggest that you use a directory at the root of a drive, like *C:\\VeloView-build*
4. open the appropriate command prompt:
`Windows Start Menu > Visual Studio 2015 > "VS2015 x86 x64 Cross Tools Command Prompt"`
* tip: for the next steps it is possible to copy a command and then past it inside the prompt with shift+insert or right-click
* this command prompt has some environnement variables set correctly to allow building (compiler path, etc)
* alternatively it is possible to use a standard cmd.exe windows prompt (in which Administrative priviledges should not be enabled for security) by entering the command: `"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" x86_amd64`
5. inside the command prompt, go to the build directory you created in step 3 by entering the command
`cd /d "C:\VeloView-build"`
* Adapt the path to your own build directory created in step 3.
* `/d` is allows to `cd` to a directory that is not on the same drive as your current path
6. inside the command prompt configure the build by entering:
`cmake <work-directory>\Veloview-source\Superbuild -GNinja -DCMAKE_BUILD_TYPE=Release -DUSE_SYSTEM_qt5=True -DQt5_DIR="C:/Qt/Qt5.10.0/5.10.0/msvc2015_64/lib/cmake/Qt5"`
* Take note that this command mentions the subdirectory "Superbuild" inside the source directory and not the source directory itself.
* Take note that the Qt5_DIR path must use **forward slashes** (like if it was an Unix PATH), because MSVC would otherwise take "\\Q" as a build option.
* If you changed the default Qt installation path, you will have to adapt this command.
* You can use absolute or relative path to point to the VeloView source directory.
* Should you want to build in RelWithDebInfo mode (in order to attach a debugger for instance), replace "Release" by "RelWithDebInfo".
* This command should show no errors, else they must be fixed.
8. inside the command prompt, start building by entering:
`ninja`
* Building from scratch can take from 45 minutes to 3 hours depending on your hardware.
* By default ninja will use all cores on your machine, but you can restrict the number of cores used by using `ninja -jN` (replace N by the number of cores to use).
9. if you modified only VeloView and want to rebuild incrementally (incrementaly = only modified files are rebuilded), enter the commands:
`cd common-superbuild/veloview/build`
`ninja install`
* Incremental builds are much faster than the first build.
* it is also possible to run `ninja` from the top of the build directory like you did the first time, but that will take longer because all dependencies are checked for changes
### Linux dependencies <a name="linux-dependencies"></a>
The following packages are needed to build on Ubuntu 16.04:
- build-essential
- cmake
- git
- flex
- byacc
- python-minimal
- python2.7-dev
- libxext-dev
- libxt-dev
- libbz2-dev
- zlib1g-dev
- freeglut3-dev
### Linux build instructions <a name="linux-build-instructions"></a>
1. clone VeloView's source code repository to a directory of your chosing, for example:
`cd <work-directory>`
`git clone <git url to VeloView repository> VeloView-source`
* moving this directoy in the future will break all build environnements that were using it (you will have to redo steps 6. and 7.)
2. clone the submodule that is used to provide the Superbuild:
`cd VeloView-source`
`git submodule update --init Superbuild/common-superbuild`
* this command must be executed inside the previously cloned repository
3. create a new directory to store the build.
`mkdir <work-directory>/VeloView-build`
* **This directory must not be inside the VeloView source code directory**
4. configure the build by entering:
`cd <work-directory>/VeloView-build`
`cmake <work-directory>/VeloView-source/Superbuild -DCMAKE_BUILD_TYPE=Release`
* by default the generator used is **make**, if you prefer to use **ninja**, add the option `-GNinja`
* Take note that this command mentions the subdirectory "Superbuild" inside the source directory and not the source directory itself.
5. start building by entering:
`make -j<N>`
* replace `<N>` by the number of cores you want to use
9. if you modified only VeloView and want to rebuild incrementally (incrementaly = only modified files are rebuilded), enter the commands:
`cd common-superbuild/veloview/build`
`make install`
* Incremental builds are much faster than the first build.
* it is also possible to run `make` from the top of the build directory like you did the first time, but that will take longer because all dependencies are checked for changes
File suppressed by a .gitattributes entry or the file's encoding is unsupported.
========================
VeloView Developer Guide
========================
---------------------------------------------------------------------------
Software developer documentation for the VeloView application and libraries
---------------------------------------------------------------------------
:copyright: Copyright (c) 2013, Velodyne Lidar. All rights reserved.
:version: 1.0.8
.. contents:: Table of Contents
.. section-numbering::
.. target-notes::
Build instructions
==================
Superbuild overview
-------------------
VeloView uses a cmake *superbuild* to download and compile third party projects
that are dependencies of VeloView. The superbuild will give you the option
to use system installations of third party projects instead of compiling them
as a superbuild step. Some dependencies, on certain platforms, must be compiled
by the superbuild, and there is no option to use a system version.
Getting the source code
-----------------------
The VeloView software is hosted in a git repository on github.com Use the
following command line to checkout the source code::
git clone git://public.kitware.com/VeloView.git
External dependencies
---------------------
The VeloView application and libraries have several external library dependencies.
As explained in the `Superbuild overview`_, the dependencies will be downloaded
and compiled automatically during the build step. See `Configure and build instructions`_.
pcap
~~~~
The required pcap version is 1.4. On Mac/Linux, only libpcap is required. On
Windows, we use the Winpcap project which includes libpcap but also includes Windows
specific drivers. Since the winpcap project only provides Visual Studio project
files, which may be out dated, the superbuild does not attempt to compile winpcap.
Instead, a git repository containing headers and precompiled .lib and .dll files
is used. The repository url is https://github.com/patmarion/winpcap.
Boost
~~~~~
The required boost version is 1.50. Boost is used for threading and synchronization,
and for network communication. The boost thread and asio libraries are used.
Qt
~~
The required Qt version is 4.8. Qt is a desktop widget library that is used
to provide user interface elements like windows and menus across the supported
platforms Windows, Mac, and Linux.
Python
~~~~~~
The required Python version is 2.7. VeloView uses libpython to embed a Python
interpreter in the VeloView application. The core VeloView features are implemented
in C++ libraries, and the libraries are wrapped for Python using VTK's Python wrapping tools.
PythonQt
~~~~~~~~
PythonQt is used from a specific git commit sha1, but a specific released version.
PythonQt is used to build Qt applications using Python. PythonQt has support
for wrapping types derived from Qt objects and VTK objects.
VTK and ParaView
~~~~~~~~~~~~~~~~
The required VTK version is 6.0. The required ParaView version is 4.0. The
ParaView repository includes VTK, so the superbuild only needs to checkout
and build ParaView in order to satisfy both dependencies. A specific git commit
sha1 is used instead of a specific released version. The commit sha1 is very similar
to the version 4.0 release but it has a few commits from the ParaView master branch
cherry-picked onto it. The commits added are those that resolve some issues with
the Python console and add the PythonQtPlugin for ParaView. The PythonQtPlugin
is a small plugin that initializes the PythonQt library and makes it available
in the ParaView Python console.
Configure and build instructions
--------------------------------
The superbuild requires cmake version 2.8.8. The build will be performed in
a separate, out-of-source build directory. Start a terminal in the parent
directory that contains your veloview checkout. Create a new build directory
and then use cmake to configure the superbuild:
mkdir build
cd build
cmake -DENABLE_veloview:BOOL=ON ../veloview/SuperBuild
This will configure the veloview superbuild with the VeloView application enabled,
and all other options left at default. For most builds, the default
options are satisfactory. Some users may wish to configure custom cmake options
in order to select system versions of certain third party dependencies such as
Qt, Boost, or Python. You should not use a system version of pcap, instead, let
the superbuild checkout the correct version of pcap or winpcap.
You can use cmake, ccmake (a curses interface to cmake) or cmake-gui.
You can set the CMAKE_BUILD_TYPE to Release, Debug, or RelWithDebInfo to set
the build type. Most users will want to select Release.
You can set the CMAKE_INSTALL_PREFIX to determine where the VeloView binaries
are installed when you run make install.
After cmake has generated the build files, just run make to run the superbuild:
make
On Mac and Linux computers, you can run parallel make with *make -j*. Parallel
make is not supported on Windows because the Windows build uses NMake.
Windows build instructions
~~~~~~~~~~~~~~~~~~~~~~~~~~
Because the superbuild compiles Python, and Python only supplies Visual Studio
project files for Visual Studio 9, you must use Visual Studio 9 for compiling
the VeloView superbuild on Windows. If you decide to use a system install of
Python instead, then you can avoid the Visual Studio version requirement. But,
be warned that other versions of Visual Studio have not been tested with VeloView.
You can build VeloView for 32bit or 64bit. The target architecture is decided
by the command prompt environment that is used when running CMake. Make sure to
open the command prompt by opening the Visual Studio Tools command prompt.
When selecting the generator in cmake-gui on Windows, you should select NMake Makefiles
using the default native compilers. It is possible to make a 32bit build on a 64bit
Windows computer by opening the Visual Studio Tools command prompt that is initialized
for the 32bit compiler environment.
After generating NMake Makefiles, just run *make* to run the superbuild. NMake
does not support parallel builds, so the build can take quite some time to complete
on Windows, especially when compiling Qt instead of using a system install of Qt.
Mac build instructions
----------------------
For Mac builds, it is best to use system installs of Qt and Python. You can use
a package manager like Homebrew or macports to install these libraries system wide
prior to compiling VeloView. The system version of pcap on Mac is too old
to be used with VeloView, so the superbuild will always download and compile the
correct version of pcap. You can choose to build Boost with the superbuild or
use a system version of Boost, as long as the static Boost archive libraries
are available (the libraries with the .a extension). If you are unsure, it is
better to let the superbuild build Boost for you.
Linux build instructions
------------------------
These steps are for Ubuntu 14.04.4 LTS. First, install the following
dependencies using the apt-get command. This is the full list used in the tested
setup::
git
cmake-curses-gui
build-essential
libboost-all-dev
libxt-dev
libbz2-dev
libqt4-dev
qt4-default
qt4-dev-tools
zlib1g-dev
By default, Ubuntu packages version 2.8 of CMake. You will need a later version.
To upgrade, either build CMake from source or run the following::
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:george-edison55/cmake-3.x
sudo apt-get update
sudo apt-get install --reinstall cmake
This will install CMake 3.2, which is sufficient to build VeloView.
On Linux, libpcap can either be installed as a package or built from source. If
you wish to build it from source you will need to apt-get install flex and
byacc. If you want to use the packaged version, apt-get install libpcap-dev. If
you're unsure, build it from source.
Clone the git repository, and from a separate build directory configure using
CMake by pointing it at the Superbuild directory::
ccmake <VeloView>/Superbuild
Enable the following options in the CMake configuration::
ENABLE_veloview=ON
USE_SYSTEM_boost=ON
USE_SYSTEM_python=ON
USE_SYSTEM_qt=ON
Also be sure to set the `USE_SYSTEM_pcap` to the appropriate value, depending on
whether you chose to use the system package or build it from source.
If you have both Qt4 and Qt5 on your system, you will need to choose Qt4 before
building. To do that, run the following in bash::
export QT_SELECT=qt4
Then run `make` or `make -jN` as usual to run the superbuild.
Packaging
---------
After building VeloView, the application will be runnable on your system. In order
to distribute VeloView to other users you must generate a VeloView package. The
packaging process is different for different platforms.
Packaging for Windows
~~~~~~~~~~~~~~~~~~~~~
Packaging on Windows requires NSIS. Visit the NSIS website to download and install
the latest version. NSIS is used to generate a standard Windows installer executable
which will install VeloView to the Program Files directory. Make sure you install
NSIS before configuring VeloView with CMake. After the superbuild has completed
(you ran make and it completed without errors) you are ready for packaging.
Before packaging, you might want to test the VeloView install tree. You can run
the make install command (make sure you have set the CMAKE_INSTALL_PREFIX to a
writable location) and then cd to the install directory and open
bin/VeloView.exe. If there are any issues, you should debug them at this point
before continuing with the packaging. Make sure you open the VeloView Python console to make sure there
are no issues with Python initialization.
To generate a Windows installer, run the package command:
make package
The output will be a .exe installer in the current directory.
Packaging for Mac
~~~~~~~~~~~~~~~~~~~~
Packaging on Mac will generate a .dmg image file. Opening the .dmg file will
mount a volume that contains the VeloView.app bundle. There is already
a VeloView.app bundle in your build tree, but it only contains the veloview
binary and not any dependent libraries. A real app bundle contains library
files for all the veloview dependencies. After copying the dependent library
files into the app bundle, a script runs the Mac tool called install_name_tool
to rewrite the library dependency locations using relative paths. The script
is in the veloview repo named fixup_bundle.py and it is executed automatically
during installation and packaging.
Before packaging, you might want to test the VeloView install tree. You
can run the make install command (make sure you have set the CMAKE_INSTALL_PREFIX
to a writable location) and then cd to the install directory and open VeloView.app.
If there are any issues, you should debug them at this point before continuing with
the packaging. Make sure you open the VeloView Python console to make sure there
are no issues with Python initialization.
To generate a Mac installer, run the package command:
make package
The output will be a .dmg file in the current directory.
Developer Guide
===============
Source code organization
------------------------
The VeloView source code is a mixture of VTK classes and Qt classes. The
source code files with the *vtk* prefix are VTK classes that do not have
any Qt dependencies. The classes with the *vv* or *pq* prefixes are Qt
classes that depend on VTK, Qt, and ParaView's Qt libraries. The core VTK
classes in VeloView are compiled into a plugin library named *libVelodyneHDLPlugin*
that can be loaded into ParaView. The VeloView app is implemented using a mixture
of the C++ Qt classes and Python code. The Python code is mostly organized in
the file *applogic.py* in the veloview Python module.
ParaView plugin
---------------
The *libVelodyneHDLPlugin* library depends on VTK, ParaView, Qt, PythonQt, Boost,
and libpcap. The plugin can be loaded into the main ParaView application using
ParaView version 4.0. The build specifies the static version of the boost libraries,
so the plugin's only dependencies beyond ParaView are libpcap and PythonQt library.
On Windows, the plugin can be loaded as long as the libpcap and PythonQt library
dll files are in the same directory. On Mac, you should use the install_name_tool
to fix the library locations of these dependencies to be relative to @loader_path,
then place the libpcap and PythonQt library files relative to the libVelodyneHDLPlugin
library.
In ParaView, the Velodyne pcap reader and Velodyne network source plugin are available
in the *Sources* menu.
VTK readers
-----------
VeloView, and the VelodyneHDL Plugin for ParaView included two readers/sources.
The Velodyne pcap reader is implemented in the C++ class vtkVelodyneHDLReader.{cxx,h}.
When reading a pcap file, the reader first scans the file and looks for frame splits
when the azimuth resets from 360 degrees to 0 degrees. The pcap file position is
recorded for each split so that the reader can jump to frames using file seeking.
The network source reader receives UDP data packets from a Velodyne sensor using
the Boost asio library. The network source is implemented by vtkVelodyneHDLSource.{cxx,h}.
The source manages multiple threads in a producer/consumer model, and uses an instance
of the vtkVelodyneHDLReader to convert data packets into VTK point cloud data.
VeloView application
--------------------
The VeloView application is implemented using Qt in C++ and Python. The PythonQt
library is used to access the C++ layer from Python. The majority of the application
logic is implemented in Python in the *applogic.py* file. The Python code also
uses Qt's uitools library to load user interface *.ui* files at runtime. Qt designer
can be used to edit the .ui files. The VeloView application can be extended using
Python and .ui files.
......@@ -78,4 +78,4 @@ HDL-64E sensor.
# How to build
Detailed instructions for building and packaging are available in the
[VeloView Developer Guide](Documentation/VeloView_Developer_Guide.pdf) .
[VeloView Developer Guide](Documentation/VeloView_Developer_Guide.md) .
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment