.. include:: ../macros.hrst
.. include:: ../abbreviations.hrst

.. _chapter:AdvancedRendering:

Advanced rendering
******************

|ParaView|'s primary focus area is quantitative analysis, wherein, conveying results objectively
with clear and simple graphs and renderings is of utmost importance. A secondary focus area is the
communication and presentation of results to a wider non-scientific audience. For this reason, there
are a specific UI elements that give you fine-grained control over precisely how the data is
displayed. One of them, :guilabel:`Enable Ray Tracing` and the related options help achieve
photo-realistic renderings of the data.

The basics
==========

*Photo-realistic rendering* of data is the ability of a rendering engine to create an image that
accurately mimics a real-life photograph of the scene. In other words, the rendering is an accurate
simulation of the physics of light and how light interacts with matter. *Ray tracing* is an
algorithm for photo-realistic rendering, that follows the paths of billions of light rays through a
scene as they interact with and bounce off objects in the environment.

Before discussing ray tracing and photo-realistic rendering, it would help the reader to understand
the different rendering approaches in computer graphics.

**Rasterization** is the task of converting 3D shapes / geometry into an image on the screen -
called a *raster image*. For each element in the scene, the primitive geometry is projected onto a
2D plane that represents the screen buffer. Different attributes like position, texture, normals,
etc. are combined using pixel shaders to determine the final color. Coupled with depth testing to
discard occluded polygons and specialized hardware in modern GPUs, rasterization is fast and
real-time. However, it does not consider physical scene parameters like global illumination,
material properties, shadows, etc. and hence cannot produce a photo-realistic image.

**Ray casting**, on the other hand, projects a ray from each pixel on the 2D screen through the 3D
shapes / geometry until it intersects a primitive that is fully opaque. The final color for each
pixel is the accumulation of the individual contributions of geometry elements that the ray
intersected with. Physical attributes and material properties of each object are accounted for, in
the calculation to come up with the final color. Different accumulation functions when casting rays
through the scene can produce different images of the scene. The cost of ray casting is directly
correlated with the number of pixels on the screen. Inherently, the information provided by
*casting* a single ray per pixel is limited and does not account inter-object interactions in the
physical scene. The produced image comes close but is *not* photo-realistic.

**Ray tracing** takes ray casting a step further by following, not only the original ray from the 2D
screen, but also all the secondary rays that will be produced due to the physics of interaction of
the light ray with an object. The primary ray and its derivatives are *traced* through the scene
until they reach a light source or they satisfy a predefined depth condition. Ray tracing captures
the visual fidelity of most light phenomena like reflections, refractions, shadows, indirect
illumination, etc. and hence produces a near accurate photo-realistic rendering. Compared to ray
casting, ray tracing is slow because after one ray bounces off an object, it can turn into 10 rays,
and those can turn into 100, 1000, and so on. This exponential increase in the number of
computations required to render a single frame, makes it time consuming.

**Path tracing** is a type of ray tracing where the rays only produce a single derivative ray per
reflection. Instead of a single ray per pixel, multiple rays are projected from the 2D screen per
pixel but these rays do not generate secondary rays in the scene. The rays are traced through the
scene until they reach a light source or a predefined depth condition. The final pixel color is an
average of all the color values accumulated by the rays projected from that pixel. If the number of
rays emitted per pixel falls short, the final image can be spotty in appearance as many pixels fail
to find any light sources from their rays. The final rendering can be improved by increasing the
number of samples (rays) per pixel. Given enough samples per pixel, path tracing can produce a
photo-realistic rendering of the scene.

Rendering backends
====================

|ParaView| uses OpenGL as the default rendering engine for displaying 3D data on the screen. In
addition, it supports NVIDIA OptiX™ and Intel\ :sup:`®`  OSPRay libraries as alternative
backends for photo-realistic rendering.

Rasterization to path tracing
=============================

Let us start by comparing the appearance of the *Can* dataset using three rendering algorithms in
|ParaView|.

Load the ``can.ex2`` file using the ``Open File`` dialog. Click :guilabel:`Apply`, change to the 8th
timestep using the :guilabel:`Time` controls in the toolbar and rotate the camera so that we get an
interesting view of the dataset. The image you see in the RenderView is created via rasterization.

.. figure:: ../images/CanRasterization.png
    :name: fig:CanRasterization
    :width: 80%
    :align: center

    Rasterization with |ParaView|

Next click :guilabel:`Enable Ray Tracing` checkbox at the bottom of the View section of the
``Properties`` panel to display the different ray tracing controls.

.. figure:: ../images/RayTraceViewControls.png
    :name: fig:RayTraceViewControls
    :width: 80%
    :align: center

    ``Properties`` panel in |ParaView| showing the major components for ray tracing

Change the :guilabel:`Back End`
dropdown to :guilabel:`OSPRay raycaster`. The image that you now see was
produced via the Ray Casting algorithm. The appearance of the object should show very little change.
Without secondary rays, ray tracing and basic rasterization both compute only local illumination
effects, where the color shown is essentially color emitted by the object itself.

.. figure:: ../images/CanRayCast.png
    :name: fig:CanRayCast
    :width: 80%
    :align: center

    Ray casting in |ParaView|

Next, check :guilabel:`Shadows`. This will send secondary rays that travel from the hit points in
the scene toward point lights in the synthetic world. Wherever these shadow rays hit an object
before the light, the color at the pixel is diminished, giving a hard shadow effect.

.. figure:: ../images/CanWithShadows.png
    :name: fig:CanWithShadows
    :width: 80%
    :align: center

    Shadows with ray casting

Next change the :guilabel:`Back End` to either :guilabel:`OSPRay pathtracer` or :guilabel:`OptiX
pathtracer` if the later is available on your system. Either one extends the process with an
arbitrary number of secondary and further rays that evaluate the contribution to color from
potentially every object in the scene.

.. figure:: ../images/CanWithOSPRayPT.png
    :name: fig:CanWithOSPRayPT
    :width: 80%
    :align: center

    Path tracing with Intel\ :sup:`®` OSPRay in |ParaView|


.. figure:: ../images/CanWithOptiXPT.png
    :name: fig:CanWithOptiXPT
    :width: 80%
    :align: center

    Path tracing with NVIDIA OptiX™ in |ParaView|


Unfortunately this image suffers from a speckling or noise artifact. The red speckles on the white
can-shaped block are where secondary light rays bounced off of the can directly onto the block to
acquire a red color. In the real world, where there are many more photons than there are rays in
this calculation, the red contribution gets averaged into the contributions made by many other
sources. One way to eliminate the speckles is to simply cast more rays. Do so by setting
``Samples Per Pixel`` higher until the speckles are not noticeable.

With complex lighting, hundreds of samples per pixel might be required to reduce the noise. In this
situation rendering slows to a crawl and interaction becomes impractical. One strategy to deal with
this is to use progressive rendering. If you start |ParaView| with the ``--enable-streaming`` command
line argument, ``Progressive Passes`` works like ``Samples Per Pixel`` except that extra rays
are cast only once the mouse is released, after which images are repeatedly rendered and blended
together such that the appearance improves over time.

Another strategy is AI denoising. To use it, make sure that ``Samples Per Pixel`` is at least 4
and then turn click the ``Denoise`` flag. This sends each rendered image through Intel's
OpenImageDenoise library where pixels are intelligently blurred to preemptively remove speckles. Do
this now and compare the results with standard path tracing.

.. figure:: ../images/CanWithDenoising.png
    :name: fig:CanWithDenoising
    :width: 80%
    :align: center

One important thing to be aware of is that with the path tracer, the overall brightness of the scene
appears greater than that with either rasterization or ray casting. Turn off Ray Tracing now and
note how both the can and the block look brighter with path tracing. This is because of the fact
that secondary rays, including those that hit |ParaView|'s default blue grey background, add color to
the objects drawn. To minimize this you can darken the objects via the ``Edit Color Map`` dialog,
the background via the ``Background Color`` controls, or everything at once via the ``Light
Scale`` entry in the ``Ray Traced Rendering`` section of the Properties Panel. Set the Light Scale
to 0.5 now to make the path traced image roughly match the brightness of the rasterized one.

.. figure:: ../images/CanWithDarkness.png
    :name: fig:CanWithDarkness
    :width: 80%
    :align: center

Materials
=========

Besides more realistic light transport modeling, another way to add realism to a scene is to apply
different materials to objects. A material is a description of a surface that includes properties
such as color, shininess, opacity, sheen, and texture. Materials offer surface rendering options
beyond the basic solid coloring and pseudo-coloring options available for scientific visualization.

Materials can be imported into |ParaView| from a JSON file that contains one or more material
descriptions. Each material may link to textures stored in image files. Texture images that define
surface colors, normals, and roughness are supported, providing a way to define a wide variety of
detailed surface features not representable by the underlying mesh alone.

|ParaView| comes with an example JSON material file that contains several materials, including glass,
copper, aluminum, wood, and others. Several of these materials also reference texture image files.

Loading Materials
~~~~~~~~~~~~~~~~~

We will start with a fresh session for this exercise.

- Reset the session by selecting the ``Edit > Reset Session`` menu item from the toolbar.

- Open the file ``can.ex2``, and click \apply.

- In the ``Properties`` panel, change the ``Back End`` under the ``Ray Traced Rendering``
  section to ``OSPRay pathtracer``.

Next, we will load a material file. To load a file, use the ``File > Load Path Tracer
Materials...`` menu item. Use the file selection dialog that appears to navigate to the location
where |ParaView|'s example material is found in |ParaView|'s installation directory. On Windows, this
location is usually ``C:/Program Files/ParaView-X.Y.Z-Windows-*/materials``. On macOS, the materials
file is stored in |ParaView|'s application bundle in
``/Applications/ParaView-X.Y.Z.app/Contents/materials``. On Linux, the materials file is located in
``share/paraview-X.Y/materials`` in the directory where |ParaView| is installed. Once you have
navigated to the materials directory, select and load the file ``ospray_mats.json``.

After loading the materials file, not much will change in the |ParaView| user interface or current
visualization. The loaded materials will appear as new items in the ``Material`` combo box in the
``Properties`` panel's ``Display`` section. Click on this combo box to see the newly loaded
materials. You will see these options in the combo box: ``None`` (use the standard surface
display), ``aluminum``, ``checker alpha``, ``copper``, ``glass``, ``scratched``, ``thin
glass``, ``water``, ``wood``, and ``Value Indexed``. The default, ``None``, means that no
material will be applied to the surface and |ParaView|'s default coloring options will be used
instead.


Applying a material
~~~~~~~~~~~~~~~~~~~

In this exercise, we will apply one of these materials to an object in the pipeline.


- Select the ``can.ex2`` file in the ``Pipeline Browser``.

- In the ``Properties`` panel, find the ``Material`` property and change it to ``copper``.

- Change the ``Samples Per Pixel`` property in the ``Ray Traced Rendering`` section to 2 to
  reduce noise speckle in the image.

You should see an image similar to this one:


.. figure:: ../images/CanSolidCopper.png
    :name: fig:CanSolidCopper
    :width: 80%
    :align: center

Note the copper color of the can and flat surface. The surface is shiny, and one part of the can is
reflected in the other.

Material can be applied per object however multiple materials may be assigned to objects by mapping
a cell data array to a material. This makes use of a special material mode called ``Value Indexed``
from the ``Material`` combo box. When this mode is selected, instead of applying a single material
to an entire object, the selected cell data array in the ``Color By`` combo box will be treated as an
index into a materials table. As such, the cell data array is expected to have a small number of
integral values or strings that represent a separate material.

In ``Value Indexed`` mode, a materials table is defined by interpreting the color map for the
selected cell data array as categories. Subsequently, each cell data value is mapped to a material
name through the ``Annotations`` table in the ``Color Map Editor``. To define the mapping from
cell data array value to material, we set the value in the ``Annotations`` table to the value of
the cell data, and the annotation to the name of the material as it appears in the ``Materials``
combo box.

In the next exercise, we will create a mapping from cell data value to material and use it to color
two different parts of the ``can.ex2`` by different materials.

Multiple materials per object
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- In the ``Properties`` panel, change the ``Material`` to ``Value Indexed``.

- Make sure the dataset color array is set to ``vtkBlockColors``.

- Open the ``Color Map Editor``. Make sure the ``Interpret Values as Categories`` checkbox is
  checked.

- Clear out all but two entries in the ``Annotations`` table by clicking on a row and clicking
  the toolbar button with the minus sign.

- In the first row, set the ``Value`` to 0 and the ``Annotation`` to ``scratched``.

- In the second row, set the ``Value`` to 1 and the ``Annotation`` to ``wood``.

When finished, the ``Color Map Editor`` should look like


.. figure:: ../images/MaterialsColorMapEditor.png
    :name: fig:MaterialsColorMapEditor
    :width: 80%
    :align: center

This table means that any cell in the ``can.ex2`` whose \texttt{vtkBlockId} value is 0 will have
the ``scratched`` material and any cell whose value is 1 will have the ``wood`` material. Material
assignments made in the ``Annotations`` table will update immediately in the render view. With the
table values you set above, the rendering will look like

.. figure:: ../images/CanWithMultipleMaterials.png
    :name: fig:CanWithMultipleMaterials
    :width: 80%
    :align: center

If an invalid material name is assigned, the data will have the standard |ParaView| property and will
take on the color that appears in the leftmost column of the annotations table.

Note that the cell data array in this exercise has one unique value per block in the multiblock
dataset. There is no requirement that this be the case, however, and a single block may be colored
by as many different materials as it has unique cell data array values.
