HDF5 version 1.10.0-alpha1 released on 2016-01-18
================================================================================


INTRODUCTION

This document describes the differences between HDF5-1.8.* and
HDF5 1.10.0-alpha1 release, and contains information on the platforms
tested and known problems in HDF5-1.10.0-alpha1.


Links to HDF5 1.10.0-alpha* source code can be found on The HDF Group's
development FTP server at the following location:
     http://www.hdfgroup.org/HDF5/release/obtain5110.html
User documentation be accessed directly at this location:
     http://www.hdfgroup.org/HDF5/docNewFeatures/
For more information, see the HDF5 home page:

    http://www.hdfgroup.org/HDF5/

If you have any questions or comments, please send them to the HDF Help Desk:

    help@hdfgroup.org


CONTENTS

- New Features
- Issues Addressed in this Release
- Tested Configuration Features Summary
- More Tested Platforms
- Known Problems


New Features
============
This alpha1 release supports the following features:

    Parallel Library:
    -----------------
    - Collective Metadata I/O
    
    Calls for HDF5 metadata can result in many small reads and writes. On metadata 
    reads, collective metadata I/O can improve performance by allowing the library
    to perform optimizations when reading the metadata, by having one rank read the 
    data and broadcasting it to all other ranks.

    Collective metadata I/O improves metadata write performance through the 
    construction of an MPI derived datatype that is then written collectively 
    in a single call. For more information see
    https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesCollectiveMetadataIoDocs.html

    Library:
    --------
    - Concurrent access to HDF5 file - Single Writer/ Multple Reader (SWMR)

    The Single Writer/ Multiple Reader or SWMR feature enables users to read  
    data concurrently while writing it. Communications between the processes 
    and file locking are not required. The processes can run on the same or 
    on different platforms, as long as they share a common file system that 
    is POSIX compliant. 
    For more information, see the Single-Writer/Multiple-Reader (SWMR) 
    Documentation at  
    https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html

    - Virtual Dataset (VDS)

    The VDS feature enables data to be accessed across HDF5 files using standard 
    HDF5 objects such as groups and datasets without rewriting or rearranging 
    the data. An HDF5 virtual dataset (VDS) is an HDF5 dataset that is composed of 
    source HDF5 datasets in a predefined mapping. VDS can be used with the SWMR 
    feature. For documentation check 
    https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html

    - Persistent Free File Space Tracking

    Usage patterns when working with an HDF5 file sometimes result in wasted space 
    within the file. This can also impair access times when working with the 
    resulting files.  The new file space management feature provides strategies 
    for managing space in a file to improve performance in both of these areas. 
    For more information see 
    http://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesFileSpaceMgmtDocs.html 


    Tools:
    ------
    - h5watch 
    
    Allows users to output new records appended to a dataset iunder SWMR access 
    as it grows. The functionality is similar to the Unix user command "tail" 
    with the follow option, which outputs appended data as the file grows. 
    For more information see
    http://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html#Tools    

    - h5format_convert
    
    The tool allows users to convert the indexing type of a chunked dataset made 
    with a 1.10.x version of the HDF5 library when the latest file format is used 
    to the 1.8.x version 1 B-tree indexing type. For example, datasets created 
    using SWMR access, can be converted to be accessed by the HDF5 1.18 library 
    and tools. The tools doesn't rewrite raw data, but HDF5 metadata only. 

    High-Level APIs:
    ------
    - H5DOappend

    The function appends data to a dataset along a specified dimension.

    Other important changes:
    -----------------------

    hid_t type was changed from 32-bit to a 64-bit value.
    
    Documentation
    -------------
    See http://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html#Tools

Issues Addressed in this Release 
================================
    - h5format_convert tool didn't downgrade version of the superblock. Fixed. See
      Known Problems section.  EIP 2016-01-11.

    - It was reported that alpha0 crashed when used with multiple threads. 
      The issue exists in the HDF5 library versions 1.8 and 1.9. 
      The problem is related to a shared file pointer used in the misc. data 
      structures. The thread-safe library exposed paths in the library where 
      file pointer becomes invalid. Alpha1 release contains the fixes for the specific 
      use case as decribed in HDFFV-9643.
      We will keep working on identifying other paths in the library with similar problem
      and fixing the issue.  EIP 2016-01-15

Supported Platforms
===================
The following platforms are supported and have been tested for this release.
They are built with the configure process unless specified otherwise.

    AIX 6.1                       xlc/xlc_r 10.1.0.5
    (NASA G-ADA)                  xlC/xlC_r 10.1.0.5
                                  xlf90/xlf90_r 12.1.0.6

    Linux 2.6.32-573.3.1.el6    GNU C (gcc), Fortran (gfortran), C++ (g++)
    #1 SMP x86_64 GNU/Linux       compilers:
    (platypus)                       Version 4.4.7 20120313
                                     Version 4.8.4, Version 5.2.0
                                  PGI C, Fortran, C++ for 64-bit target on
                                  x86-64;
                                      Version 15.7-0
                                  Intel(R) C (icc), C++ (icpc), Fortran (icc)
                                  compilers:
                                     Version 15.0.3.187 Build 20150407
                                  MPICH 3.1.4 compiled with GCC 4.9.3

    Linux 2.6.32-504.8.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11)
    #1 SMP ppc64 GNU/Linux        g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11)
    (ostrich)                     GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11)
                                  IBM XL C/C++ V13.1
                                  IBM XL Fortran V15.1

    Linux 3.10.0-229.14.1.el7   GNU C (gcc), Fortran (gfortran), C++ (g++)
    #1 SMP x86_64 GNU/Linux       compilers:
    (kituo/moohan)                  Version 4.8.3 20140911 (Red Hat 4.8.3-9)
                                    Version 5.2.0
                                  Intel(R) C (icc), C++ (icpc), Fortran (icc)
                                  compilers:
                                     Version 15.0.3.187 Build 20150407
                                  MPICH 3.1.4 compiled with GCC 4.9.3

    SunOS 5.11 32- and 64-bit     Sun C 5.12 SunOS_sparc
    (emu)                         Sun Fortran 95 8.6 SunOS_sparc
                                  Sun C++ 5.12 SunOS_sparc

    Windows 7                     Visual Studio 2012 w/ Intel Fortran 15 (cmake)
                                  Visual Studio 2013 w/ Intel Fortran 15 (cmake)
                                  Visual Studio 2015 w/ Intel Fortran 16 (cmake)
                                  Cygwin(CYGWIN_NT-6.1 2.2.1(0.289/5/3) gcc(4.9.3) compiler and gfortran)
                                  (cmake and autotools)

    Windows 7 x64                 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
                                  Visual Studio 2013 w/ Intel Fortran 15 (cmake)
                                  Visual Studio 2015 w/ Intel Fortran 16 (cmake)

    Windows 8.1                   Visual Studio 2012 w/ Intel Fortran 15 (cmake)
                                  Visual Studio 2013 w/ Intel Fortran 15 (cmake)

    Windows 8.1 x64               Visual Studio 2012 w/ Intel Fortran 15 (cmake)
                                  Visual Studio 2013 w/ Intel Fortran 15 (cmake)

    Mac OS X Mt. Lion 10.8.5      Apple clang/clang++ version 5.1 from Xcode 5.1
    64-bit    			  gfortran GNU Fortran (GCC) 4.8.2
    (swallow/kite)                Intel icc/icpc/ifort version 15.0.3

    Mac OS X Mavericks 10.9.5     Apple clang/clang++ version 6.0 from Xcode 6.2.0
    64-bit    			  gfortran GNU Fortran (GCC) 4.9.2
    (wren/quail)                  Intel icc/icpc/ifort version 15.0.3

    Mac OS X Yosemite 10.10.5     Apple clang/clang++ version 6.0 from Xcode 7.0.0
    64-bit    			  gfortran GNU Fortran (GCC) 4.9.2
    (osx1010dev/osx1010test)      Intel icc/icpc/ifort version 15.0.3


Tested Configuration Features Summary
=====================================

    In the tables below
          y   = tested
          n   = not tested in this release
          C   = Cluster
          W   = Workstation
          x   = not working in this release
          dna = does not apply
          ( ) = footnote appears below second table
          <blank> = testing incomplete on this feature or platform

Platform                              C         F90/   F90      C++  zlib  SZIP
                                      parallel  F2003  parallel
SunOS 5.11 32-bit                       n        y/y    n        y    y     y
SunOS 5.11 64-bit                       n        y/y    n        y    y     y
Windows 7                               y        y/y    n        y    y     y
Windows 7 x64                           y        y/y    n        y    y     y
Windows 7 Cygwin                        n        y/y    n        y    y     n
Windows 8.1                             n        y/y    n        y    y     y
Windows 8.1 x64                         n        y/y    n        y    y     y
Mac OS X Mountain Lion 10.8.5 64-bit    n        y/y    n        y    y     y
Mac OS X Mavericks 10.9.5 64-bit        n        y/y    n        y    y     y
Mac OS X Yosemeti 10.10.5 64-bit        n        y/y    n        y    y     y
AIX 6.1 32- and 64-bit                  n        y/n    n        y    y     y
CentOS 6.7 Linux 2.6.32 x86_64 GNU      y        y/y    y        y    y     y
CentOS 6.7 Linux 2.6.32 x86_64 Intel    n        y/y    n        y    y     y
CentOS 6.7 Linux 2.6.32 x86_64 PGI      n        y/y    n        y    y     y
CentOS 7.1 Linux 3.10.0 x86_64 GNU      y        y/y    y        y    y     y
CentOS 7.1 Linux 3.10.0 x86_64 Intel    n        y/y    n        y    y     y
Linux 2.6.32-431.11.2.el6.ppc64         n        y/n    n        y    y     y

Platform                                 Shared  Shared    Shared    Thread-
                                         C libs  F90 libs  C++ libs  safe
SunOS 5.11 32-bit                          y       y         y         y
SunOS 5.11 64-bit                          y       y         y         y
Windows 7                                  y       y         y         y
Windows 7 x64                              y       y         y         y
Windows 7 Cygwin                           n       n         n         y
Windows 8.1                                y       y         y         y
Windows 8.1 x64                            y       y         y         y
Mac OS X Mountain Lion 10.8.5 64-bit       y       n         y         y
Mac OS X Mavericks 10.9.5 64-bit           y       n         y         y
Mac OS X Yosemeti 10.10.5 64-bit           y       n         y         y
AIX 6.1 32- and 64-bit                     y       n         n         y
CentOS 6.7 Linux 2.6.32 x86_64 GNU         y       y         y         y
CentOS 6.7 Linux 2.6.32 x86_64 Intel       y       y         y         y
CentOS 6.7 Linux 2.6.32 x86_64 PGI         y       y         y         y
CentOS 7.1 Linux 3.10.0 x86_64 GNU         y       y         y         y
CentOS 7.1 Linux 3.10.0 x86_64 Intel       y       y         y         y
Linux 2.6.32-431.11.2.el6.ppc64            y       y         y         y

Compiler versions for each platform are listed in the preceding
"Supported Platforms" table.


More Tested Platforms
=====================
The following platforms are not supported but have been tested for this release.

    Linux 2.6.18-431.11.2.el6     g95 (GCC 4.0.3 (g95 0.94!)
    #1 SMP x86_64 GNU/Linux
    (platypus)

    Mac OS X El Capitan 10.11     Apple clang/clang++ version 7.0.0 from Xcode 7.0.1
    64-bit    			  gfortran GNU Fortran (GCC) 5.2.0
    (VM)

    Windows 7                     Visual Studio 2008  (cmake)

    Windows 7 x64                 Visual Studio 2008  (cmake)

    Windows 10                    Visual Studio 2013 w/ Intel Fortran 15 (cmake)

    Windows 10 x64                Visual Studio 2013 w/ Intel Fortran 15 (cmake)

    Debian7.5.0 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux
                                  gcc (Debian 4.7.2-5) 4.7.2
                                  GNU Fortran (Debian 4.7.2-5) 4.7.2
                                  (cmake and autotools)

    Fedora20 3.15.3-200.fc20.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
                                  gcc (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1)
                                  GNU Fortran (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1)
                                  (cmake and autotools)

    SUSE 13.1 3.11.10-17-desktop #1 SMP PREEMPT x86_64 x86_64 x86_64 GNU/Linux
                                  gcc (SUSE Linux) 4.8.1
                                  GNU Fortran (SUSE Linux) 4.8.1
                                  (cmake and autotools)

    Ubuntu 14.04 3.13.0-35-generic #62-Ubuntu SMP x86_64 GNU/Linux
                                  gcc (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1
                                  GNU Fortran (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1
                                  (cmake and autotools)

    hopper.nersc.gov              PrgEnv-gnu/5.2.40
                                  gcc (GCC) 4.9.2 20141030 (Cray Inc.)
                                  GNU Fortran (GCC) 4.9.2 20141030 (Cray Inc.)
                                  g++ (GCC) 4.9.2 20141030 (Cray Inc.)


Known Problems
==============
WARNINGS: This section is under contsruction. It will be updated for the first HDF5 1.10.0-beta
release. 

  - h5format_convert only works with the files that contain chunked datasets.
  - h5repack tool fails to downgrade object layout message to make file 
    1.8 compatible.

