VTKExamples Administrators have write access to the git repository. If you are a VTKExamples User go here or a VTKExamples Developer go here.

Organization of the VTKExamples Repository¶

The VTKExamples are stored in a git repository hosted at github.com. The repository contains several types of files.

All example source code, descriptions, test data and test baselines are stored in the src/ tree.

VTKExamples Repository¶

The major elements of the tree are:

|-- VTKExamples
  |-- CMakeLists.txt   # To build all of the examples
  |-- _layouts         # Contains _default.html
  |-- custom_theme     # tweaks to material theme
  |-- mkdocs.yml       # config file for MkDocs
  |-- src              # all of the content to create site/
    |-- SyncSiteWithRepo.sh  # master script to update site/
    |-- Admin
    |   |-- ScrapeRepo       # script to create docs/
    |   |-- VTKCMakeLists    # template for Cxx examples
    |   |-- VTKQtCMakeLists  # template for Cxx Qt
    |   |-- sitemap.xml      # generated sitemap
    |   |-- sitemapGenerator # site map generator
    |-- Artifacts       # Additional images, docs, etc.
    |-- Instructions    # Documentation
    |-- LANGUAGE.md     # Summary file for language
    |-- LANGUAGE        # Language examples
    |   |-- TOPIC
    |-- Tarballs     # Tar files for each Cxx example
    |-- Testing
    |   |-- Baseline # Baselines for examples
    |   |   |-- LANGUAGE
    |   |   |   |-- TOPIC
    |   |-- Data    # Data for examples
    |-- stylesheets
        |-- extra.css # mkdocs search config
  |-- site  # static web pages generated by mkdocs

Look and Feel¶

A priority in moving from the wiki media VTK Wiki Examples to the github pages VTKExamples was to provide a modern, familiar look and feel. We also wanted to support tablet and mobile platforms. We chose MkDocs because it generated static html pages that can be hosted anywhere.

MkDocs¶

Installing MkDocs¶

MkDocs is a python package and can be installed using pip;

pip install mkdocs

Configuring MkDocs¶

The mkdocs.yml file contains the configuration parameters

site_name: VTKExamples
theme: material
site_url: https://lorensen.github.io/VTKExamples/site/
repo_name: lorensen/VTKExamples
repo_url: https://github.com/lorensen/VTKExamples
theme_dir: custom_theme
markdown_extensions:
  - admonition
  - codehilite(guess_lang=false)
  - toc(permalink=true)
google_analytics:
  -  'UA-3660134-4'
  -  'auto'
extra_css:
  -  'stylesheets/extra.css'
copyright: '<font color=#666>Content is available under </font><a href="https://creativecommons.org/licenses/by/2.5/" title="Attribution2.5">Attribution2.5</a> <font color=#666>unless otherwise noted.</font>'

Markdown Extensions¶

A number of markdown extensions are available.

admonition¶

Admonitions are specially marked "topics" that can appear anywhere an ordinary body element can. They contain arbitrary body elements. Typically, an admonition is rendered as an offset block in a document, sometimes outlined or shaded, with a title matching the admonition type. For example:

codehilite¶

The codehilite extension highlights code depending on the language.

toc¶

The toc extension generates a table of contents, like the one at the right of this page.

MkDocs Materials theme¶

The Materials theme is built using Google's Material Design guidelines. It offers a unified experience across platforms. The look and feel is a familiar one.

Configuring Materials¶

The materials theme is selected using the material keyword in the mkdocs.yml file.

Google Analytics¶

Google Analytics tracks the site usage, providing lots of useful statistics. To have Google Analytics track web usage you need to register the url with the Google Analytics System. After registration, an html snippet is provided to include on every web page to be tracked.

Configuring Google Analytics¶

The google_analytics keyword in the mkdocs.yml" file specifies the google analytics unique code for this web site. The custom_theme/main.html* file defines the meta data for the google site verification:

<meta name="google-site-verification" content="8hi6AHNlzKOmFDV4W8tLmySODRzQvtqMEIfZdc3WTLA" />

Google Custom Search Engine¶

The overall look and feel is established at https://cse.google.com/cse/. After setting up the search engine, you can get the code to add to the web pages.

<script>
  (function() {
    var cx = '015419652359195128492:0lkxso1wcym';
    var gcse = document.createElement('script');
    gcse.type = 'text/javascript';
    gcse.async = true;
    gcse.src = 'https://cse.google.com/cse.js?cx=' + cx;
    var s = document.getElementsByTagName('script')[0];
    s.parentNode.insertBefore(gcse, s);
  })();
</script>

Configuring GCSE¶

The code is added to custom_theme/main.html.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<gcse:searchbox-only>Search</gcse:searchbox-only>

Implementing GCSE¶

The search box is added to the web pages by adding the gcse search box html

    <gcse:searchbox-only>Search</gcse:searchbox-only>

Performance¶

Lazy Image Loading¶

The first implementation had problems loading the Cxx summary. The number of embedded images exceeded the throttle limits of github. A lazy image load solution solved the problem. Now, images are only loaded if they appear in the browser window.

The Lazy Image Loading is implemented in javascript. The javascript is stored in custom_theme/main.html. If uses a clever trick. Images that should be lazy loaded use a tag with a class=lazy and attribute data_src. If the image is in the view port, the data_src is removed and the image renders.

Tiny URLs¶

The VTKExamples web pages have many url's that link to images and VTK doxygen pages. The long length of these url's can increase page load tines, especially on the language summary pages.

http://tinyurl.com provides a service to map the long url's into shorted url's. There is a python package, tinyurl, that converts the long url's. Unfortunately, this process is slow. To speed up the process, we cache the long/short url's in a file src/Admin/TinyUrlCache. ScrapeRepo uses this cache to only convert url's that are not in the cache. ScrapeRepo updates the cache after each run.

tinyurl must be installed:

pip install tinyurl

Minify HTML pages¶

mkdocs generates HTML pages from the markdown files in doc. These pages have lots of white space. We use the python package htmlmin to compress the generated HTML. The compressed pages are abput 30% smaller after running htmlmin. We use the command line to process each index.html file in site/.

htmlmin must be installed:

pip install htmlmin

Administrator Tasks¶

ScrapeRepo¶

Given a RepositoryDir, DocDir and a RepositoryUrl, ScrapeRepo proceeds as follows:

1. Adds thumbnails to the Cxx.md and Python.md summary pages. A thumbnail is added if a src/Testing/Baseline/LANG/TOPIC/TestEXAMPLE.png image exists for the example.

2. Copies non-example .md files like Cxx.md and Python.md files to docs/.

3. For each example source file create docs/LANG/TOPIC/EXAMPLE.md file that contains:

   1. If src/Testing/Baseline/LANG/TOPIC/TestEXAMPLE.png exists add the image
   2. If a src/LANG/TOPIC/EXAMPLE.md description file exists, insert the description
   3. Insert the source code from src/LANG/TOPIC/EXAMPLE.EXTENSION

4. For each Cxx source file

   1. If Cxx/TOPIC/EXAMPLE.cmake exists, add it as the CMakeLists section of the .md file, otherwise, fill in the VTKCMakelists template, including the files listed in the optional src/Cxx/TOPIC/EXAMPLE.extras file.

5. For each Cxx source file, create a src/Tarballs/EXAMPLE.tar file containing the source and CMakeLists.txt file.

SyncSiteWithRepo¶

The SyncSiteWithRepo.sh shell script executes all of the steps to update the static website. It proceeds as follow:

1. Makes sure the site is up

2. Pulls updates from the master repository at https://github.com/lorensen/VTKExamples

3. Pulls updates from the src/Tarballs repository

   The tarballs are kept in a separate repository to reduce the size of the main repository. They are only accessed from the individual example pages.

4. Creates the coverage files

   The src/Admin/VTKClassesUsedInExamples.py python script generates two tables for each language. One table list each class and what classes it uses. The second table lists the classes that are not used in any example.

5. Wipes the docs directory

   The docs directory contains all of the md and html files for the site. A clean directory prevents old files from being used.

6. Runs the ScrapeRepo script

   This will populate the docs directory.

7. Checks for a successful scrape

   This sanity check sees if a reasonable number of files have been updated.

8. Copies the src/stylesheets directory ino docs/stylesheets

9. Copies the sitemap.xml file

   This file is updated periodically by sitemapGenerator

10. Minify the html

11. Process any modified, added or deleted file

12. Update the tarballs

13. Push the changes

    After a short time github will update the github pages.

How to Become an Administrator¶