# How to set up an Amazon EC2 AMI instance to run ParaViewWeb

## Introduction

This document describes how to set up a clean Amazon EC2 AMI instance to run ParaViewWeb with the following configuration:

1. Deploy the latest nightly binary version of ParaView
2. Use a recent version of Apache, which now supports web sockets
3. Use the Jetty session manager to launch python processes

In this document, we will refer to the DNS name of our EC2 host as

    ec2-XXX-XXX-XXX-XXX.compute-1.amazonaws.com

You should replace this with the actual hostname of a running instance. Additionally, whenever you secure shell into your instance, it is possible that you will get a message from the system regarding the installation of updates.  It is most likely a good idea to install the updates. To do so, follow the system instructions:

    sudo yum update

## Package install and graphics environment setup

Before proceeding with the installation specifics, you should install some required packages on the instance.  The patch package is required for patching the Apache httpd source, pcre-devel is required by the Apache compilation phase, mesa-libGLU is required to perform ParaView rendering, and the xorg packages are required for a proper X server environment.

    $ sudo yum install patch.x86_64
    $ sudo yum install pcre-devel.x86_64
    $ sudo yum install mesa-libGLU.x86_64
    $ sudo yum install xorg*

After running the above package installations, you should run nvidia-config, as shown below, so as to generate a correctly formatted X configuration file.

    $ sudo nvidia-config

After running the above nvidia-config command, there will be a new file generated: /etc/X11/xorg.conf.  This file needs to be edited to include the BusID in the "Device" section, as well as a couple of items in the "Screen" section (an "Option" line within "Screen", as well as a "Virtual" line within "Screen"->"Subsection").  The entire contents of the file (from a recent installation) are included below for reference.

    # nvidia-xconfig: X configuration file generated by nvidia-xconfig
    # nvidia-xconfig:  version 319.60  (buildmeister@swio-display-x64-rhel04-15)  Wed Sep 25 15:17:31 PDT 2013

    Section "ServerLayout"
        Identifier     "Layout0"
        Screen      0  "Screen0"
        InputDevice    "Keyboard0" "CoreKeyboard"
        InputDevice    "Mouse0" "CorePointer"
    EndSection

    Section "Files"
    EndSection

    Section "InputDevice"
        # generated from default
        Identifier     "Mouse0"
        Driver         "mouse"
        Option         "Protocol" "auto"
        Option         "Device" "/dev/psaux"
        Option         "Emulate3Buttons" "no"
        Option         "ZAxisMapping" "4 5"
    EndSection

    Section "InputDevice"
        # generated from data in "/etc/sysconfig/keyboard"
        Identifier     "Keyboard0"
        Driver         "kbd"
        Option         "XkbLayout" "us"
        Option         "XkbModel" "pc105"
    EndSection

    Section "Monitor"
        Identifier     "Monitor0"
        VendorName     "Unknown"
        ModelName      "Unknown"
        HorizSync       28.0 - 33.0
        VertRefresh     43.0 - 72.0
        Option         "DPMS"
    EndSection

    Section "Device"
        Identifier     "Device0"
        Driver         "nvidia"
        VendorName     "NVIDIA Corporation"
        BusID           "00:03:0"
    EndSection

    Section "Screen"
        Identifier     "Screen0"
        Device         "Device0"
        Monitor        "Monitor0"
        DefaultDepth    24
        Option         "UseDisplayDevice" "None"
        SubSection     "Display"
            Virtual     1920 1080
            Depth       24
        EndSubSection
    EndSection

Your choice for the "Virtual" screen resolution, located near the bottom of the file, will become the maximum size image that ParaView can render. You may want to increase the resolution accordingly.

## ParaView

This section of the document describes how to acquire all of the pieces necessary to deploy ParaViewWeb on the instance, including ParaView, the documentation, and the ParaViewData repository.

    $ cd /home/ec2-user/
    $ mkdir ParaView
    $ cd ParaView
    $ wget http://www.paraview.org/files/nightly/ParaView-Linux-64bit-glibc-2.3.6-NIGHTLY.tar.gz -O pv.tgz
    $ tar xvzf pv.tgz
    $ wget http://www.paraview.org/files/nightly/ParaView-doc.tar.gz -O pv-doc.tgz
    $ tar xvzf pv-doc.tgz
    $ sudo mkdir -p /var/www/pvweb-deploy/www
    $ sudo cp -r ParaView-4.1.0-RC1-Linux-64bit/share/paraview-4.1/www/* /var/www/pvweb-deploy/www
    $ sudo cp -r www/js-doc/* /var/www/pvweb-deploy/www
    $ cd /var/www/pvweb-deploy/www
    $ sudo mv index.html index.origin
    $ sudo sh -c 'cat index.origin | grep -v DEMO-APPS > index.html'
    $ cd /home/ec2-user/ParaView
    $ wget http://paraview.org/files/v4.0/ParaViewData-v4.0.1.tar.gz
    $ tar zxvf ParaViewData-v4.0.1.tar.gz
    $ mv ParaViewData-v4.0.1 ParaViewData

## Apache

In this section, you will learn how to download the source for Apache httpd and then build, install, and configure it for use with ParaViewWeb.

### Download, build, and install Apache httpd

General instructions for building and installing Apache 2.4.7 on the ec2 machine can be found at the following URL:

    http://httpd.apache.org/docs/2.4/install.html

However, the instructions below give detailed steps that can be followed on the Amazon EC2 AMI instance.

Obtain the necessary source tarballs. You will need httpd source, as well as apr and apr-util sources.

    $ mkdir /home/ec2-user/downloads
    $ cd /home/ec2-user/downloads
    $ wget http://mirrors.sonic.net/apache/httpd/httpd-2.4.7.tar.gz -O httpd.tgz
    $ wget http://apache.petsads.us/apr/apr-1.5.0.tar.gz -O apr.tgz
    $ wget http://apache.petsads.us/apr/apr-util-1.5.3.tar.gz -O apr-util.tgz

Now unpack everything in the right places.

    $ mkdir /home/ec2-user/apache-2.4.7-src
    $ cd /home/ec2-user/apache-2.4.7-src
    $ tar zxvf /home/ec2-user/downloads/httpd.tgz
    $ cd httpd-2.4.7/srclib
    $ tar zxvf /home/ec2-user/downloads/apr.tgz
    $ mv apr-1.5.0 apr
    $ tar zxvf /home/ec2-user/downloads/apr-util.tgz
    $ mv apr-util-1.5.3 apr-util

Note that for Apache version less than 2.4.7, you will need to patch
the sources before building:

    $ wget -O httpd-patch.txt https://issues.apache.org/bugzilla/attachment.cgi?id=30886&action=diff&context=patch&collapsed=&headers=1&format=raw
    $ cd httpd-[version]
    $ patch -u -p1 < /home/ec2-user/downloads/httpd-patch.txt

Now configure the Apache build.

    $ cd /home/ec2-user/apache-2.4.7-src/httpd-2.4.7
    $ ./configure --prefix /opt/apache-2.4.7 --with-included-apr --enable-proxy

Check how many processors are available and use them all to build httpd.  The nproc command returns the number of processors that are available. Use this number in the make command.  When the build is finished, install httpd.

    $ nproc
    $ make -j<number>
    $ sudo make install

### Configure Apache httpd for use with ParaViewWeb

Create a directory for the mapping file that Jetty and Apache use to communicate about sessions and port locations.

    $ sudo mkdir -p /opt/apache-2.4.7/pv-mapping-file
    $ sudo touch /opt/apache-2.4.7/pv-mapping-file/mapping.txt
    $ sudo groupadd mappingfileusers
    $ sudo usermod -a -G mappingfileusers ec2-user
    $ newgrp mappingfileusers
    $ sudo usermod -a -G mappingfileusers daemon
    $ sudo chgrp mappingfileusers /opt/apache-2.4.7/pv-mapping-file/mapping.txt
    $ sudo chmod 660 /opt/apache-2.4.7/pv-mapping-file/mapping.txt

Add a virtual host to the httpd-vhosts.conf file, which will be located in the directory /opt/apache-2.4.7/conf/extra/.  Make sure to replace the "ServerName" value with the correct EC2 instance DNS name.  It is probably a good idea to comment out or remove the existing example virtual host entries in this file at the same time. Otherwise, you will see errors when starting Apache.

    <VirtualHost *:80>
        ServerName ec2-XXX-XXX-XXX-XXX.compute-1.amazonaws.com
        ServerAdmin webmaster@example-host.example.com
        DocumentRoot "/var/www/pvweb-deploy/www"
        ErrorLog "logs/pv-error_log"
        CustomLog "logs/pv-access_log" common

        ProxyPass /paraview http://localhost:9000/paraview

        # Turn on the rewrite engine
        RewriteEngine On

        # This is the path the mapping file Jetty creates
        RewriteMap session-to-port txt:/opt/apache-2.4.7/pv-mapping-file/mapping.txt

        # This is the rewrite condition. Look for anything with a sessionId= in the query part of the URL and capture the value to use below.
        RewriteCond %{QUERY_STRING}     ^sessionId=(.*)$ [NC]

        # This does the rewrite using the mapping file and the sessionId
        RewriteRule    ^/proxy.*$  ws://${session-to-port:%1}/ws  [P]

        <Directory "/var/www/pvweb-deploy/www">
            Options Indexes FollowSymLinks
            Order allow,deny
            Allow from all
            AllowOverride None
            Require all granted
        </Directory>

    </VirtualHost>

Include this virtual host in the main httpd configuration file.  Find the following line in httpd.conf and uncomment it:

    Include conf/extra/httpd-vhosts.conf

Find the following lines in the httpd.conf file and uncomment them in order to load some necessary modules:

    LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
    LoadModule rewrite_module modules/mod_rewrite.so

Start the httpd daemon.

    $ sudo /opt/apache-2.4.7/bin/apachectl -k start

At this point, you can check to make sure that at least the static content is available by pointing your browser at the instance.  As an example, you can paste a URL like the following one into your browser's location bar:

    http://ec2-XXX-XXX-XXX-XXX.compute-1.amazonaws.com

Just make sure to use the correct instance DNS name.

## Configuring the pvpython launcher

The final steps in this process are configuring the launcher for the pvpython visualization processes and starting the launcher.  We will illustrate how to set up Jetty to handle this task.

Make a directory to hold the configuration file, the launcher jar, and the launcher logs.

    $ mkdir -p /home/ec2-user/ParaView/launcher/logs
    $ cd /home/ec2-user/ParaView/launcher
q
Download the launcher jar file from [here](http://paraview.org/files/dependencies/ParaViewWeb/JettySessionManager-Server-1.1.jar).  The launcher directory created in the previous step would be a good place to put the launcher jar file.  It might be more convenient to just do the following, as long as you are already in the "launcher" directory:

    $ wget http://paraview.org/files/dependencies/ParaViewWeb/JettySessionManager-Server-1.1.jar

Create a jetty configuration file by using directly or modifying the following example.  You will need to at least modify the hostname in the "pw.factory.session.url.generator.pattern" element.  The directory you created to hold the launcher logs would be a good place to put this configuration file, and you could call it "jetty-config.txt".

    # ===================================================
    # Expect a configuration file as the argument
    #
    #  $ java -jar file.jar <config_file_path>.
    #
    # The content of that file should look like this
    # ===================================================

    # Web setup
    pw.web.port=9000
    pw.web.content.dir=/var/www/pvweb-deploy/www

    # Process logs
    pw.logging.dir=/home/ec2-user/ParaView/launcher/logs

    # ==================================================
    # Process command: data_prober.py      | data_prober
    # ==================================================
    pw.data_prober.cmd=./bin/pvpython ./lib/paraview-4.1/site-packages/paraview/web/pv_web_data_prober.py -f --data-dir /home/ec2-user/ParaView/ParaViewData/Data --port PORT --authKey SECRET
    pw.data_prober.cmd.run.dir=/home/ec2-user/ParaView/ParaView-4.1.0-RC1-Linux-64bit
    pw.data_prober.cmd.map=PORT:getPort|SECRET:secret

    # ==================================================
    # Process command: file_loader.py      | loader
    # ==================================================
    pw.loader.cmd=./bin/pvpython ./lib/paraview-4.1/site-packages/paraview/web/pv_web_file_loader.py -f --data-dir /home/ec2-user/ParaView/ParaViewData/Data --port PORT --authKey SECRET
    pw.loader.cmd.run.dir=/home/ec2-user/ParaView/ParaView-4.1.0-RC1-Linux-64bit
    pw.loader.cmd.map=PORT:getPort|SECRET:secret

    # ==================================================
    # Process command: pipeline_manager.py | pipeline
    # ==================================================
    pw.pipeline.cmd=./bin/pvpython ./lib/paraview-4.1/site-packages/paraview/web/pv_web_visualizer.py -f --data-dir /home/ec2-user/ParaView/ParaViewData/Data --port PORT --authKey SECRET
    pw.pipeline.cmd.run.dir=/home/ec2-user/ParaView/ParaView-4.1.0-RC1-Linux-64bit
    pw.pipeline.cmd.map=PORT:getPort|SECRET:secret

    # Resources informations
    pw.resources=localhost:9001-9100

    # Factory
    pw.factory.proxy.adapter=com.kitware.paraviewweb.external.ApacheModRewriteMapFileConnectionAdapter
    pw.factory.session.url.generator=com.kitware.paraviewweb.external.GenericSessionURLGenerator
    pw.factory.resource.manager=com.kitware.paraviewweb.external.SimpleResourceManager
    pw.factory.visualization.launcher=com.kitware.paraviewweb.external.ProcessLauncher
    pw.factory.websocket.proxy=com.kitware.paraviewweb.external.SimpleWebSocketProxyManager
    pw.factory.session.manager=com.kitware.paraviewweb.external.MemorySessionManager

    # External configurations
    pw.factory.proxy.adapter.file=/opt/apache-2.4.7/pv-mapping-file/mapping.txt

    # CAUTION: The ws port should match the server port
    # For Jetty websocket forwarder use: ws://localhost:9000/paraview/SESSION_ID
    # For Apache front-end use         : ws://localhost/proxy?sessionId=SESSION_ID
    pw.factory.session.url.generator.pattern=ws://ec2-XXX-XXX-XXX-XXX.compute-1.amazonaws.com/proxy?sessionId=SESSION_ID

    # Timeout for websocket proxy in milliseconds
    pw.factory.websocket.proxy.timeout=300000

    pw.process.launcher.wait.keyword=Starting factory
    pw.process.launcher.wait.timeout=10000

    pw.session.public.fields=id,sessionURL,name,description,sessionManagerURL,application,idleTimeout,startTime,file
    pw.session.max=10

    # ===================================================

Once the launcher jar is downloaded and available, and once the config file is in place, it is almost time to start the launcher.  The final steps involve starting an X server (in the background), setting the DISPLAY environment variable, and running the launcher (also in the background).

    $ sudo X :0 &
    $ export DISPLAY=:0.0
    $ cd /home/ec2-user/ParaView/launcher
    $ java -jar JettySessionManager-Server-1.1.jar jetty-config.txt &

## Final Testing

Now your instance should be set up to run ParaViewWeb.  Go back to your browser and choose any of the applications linked from the ParaViewWeb page you tested above.
