sensei issueshttps://gitlab.kitware.com/sensei/sensei/-/issues2021-06-08T10:32:20-04:00https://gitlab.kitware.com/sensei/sensei/-/issues/57VTKPosthocIO and PVD files (Unstructured Grid)2021-06-08T10:32:20-04:00Victor MateevitsiVTKPosthocIO and PVD files (Unstructured Grid)I have been working on instrumenting Nek5000 with SENSEI and have stumbled on an issue.
The data is Unstructured Grid, so in my GetMeshMetadata function, I have the following:
```
metadata->MeshType = VTK_UNSTRUCTURED_GRID;
meta...I have been working on instrumenting Nek5000 with SENSEI and have stumbled on an issue.
The data is Unstructured Grid, so in my GetMeshMetadata function, I have the following:
```
metadata->MeshType = VTK_UNSTRUCTURED_GRID;
metadata->CoordinateType = VTK_DOUBLE;
metadata->NumArrays = 2;
metadata->ArrayName = {"pressure", "velocity"};
metadata->ArrayCentering = {vtkDataObject::CELL, vtkDataObject::CELL};
metadata->ArrayComponents = {1, 3};
metadata->ArrayType = {VTK_DOUBLE, VTK_DOUBLE};
metadata->StaticMesh = 0;
````
If I use catalyst, the data gets rendered correctly using SENSEI.
However, if I switch to PosthocIO analysis, while I can see the *.vtu files being generated (and I can open them individually), the PVD file is as follows:
```
<?xml version="1.0"?>
<VTKFile type="Collection" version="0.1" byte_order="LittleEndian" compressor="">
<Collection>
</Collection>
</VTKFile>
```
Debugging to see why that happens, we can see in VTKPosthocIO.cxx (line 499) the following:
```
pvdFile << "<?xml version=\"1.0\"?>" << endl
<< "<VTKFile type=\"Collection\" version=\"0.1\""
" byte_order=\"LittleEndian\" compressor=\"\">" << endl
<< "<Collection>" << endl;
for (long i = 0; i < nSteps; ++i)
{
for (long j = 0, k = 0; j < mmd[i]->NumBlocks; ++j)
{
if (mmd[i]->BlockNumCells[j] > 0)
{
std::string fileName =
getBlockFileName("./", meshName, mmd[i]->BlockIds[j], i, blockExt);
pvdFile << "<DataSet timestep=\"" << times[i]
<< "\" group=\"\" part=\"" << k << "\" file=\"" << fileName
<< "\"/>" << endl;
++k;
}
}
}
```
The problem here is the `mmd[i]->NumBlocks` (which is zero), because there are no Blocks. Is this a bug or is it expected? Do I have to convert the Nek5000 data to Multiblock?
Thanks!
PS: For future reference, for similar issues are we using the issue tracker or discourse?https://gitlab.kitware.com/sensei/sensei/-/issues/3libsim build issue2021-02-01T19:21:16-05:00Burlen Loringlibsim build issuerecall the error seen during the SENSEI tutorial at SC17:
```
c++ [options removed] -o ../../bin/3D_Grid ...[stuff removed] /local/apps/VisIt/2.13/current/linux-x86_64/libsim/V2/lib/libsimV2.a
/usr/bin/ld: /local/apps/VisIt/2.13/curre...recall the error seen during the SENSEI tutorial at SC17:
```
c++ [options removed] -o ../../bin/3D_Grid ...[stuff removed] /local/apps/VisIt/2.13/current/linux-x86_64/libsim/V2/lib/libsimV2.a
/usr/bin/ld: /local/apps/VisIt/2.13/current/linux-x86_64/libsim/V2/lib/libsimV2.a(VisItControlInterface_V2.c.o): undefined reference to symbol 'dlsym@@GLIBC_2.2.5'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libdl.so: error adding symbols: DSO missing from command line
```
I solved it by looking at how I compiled my own libsim demos. To fix it, I added:
```
-Wl,-Bstatic /local/apps/VisIt/2.13/current/linux-x86_64/libsim/V2/lib/libsimV2.a -Wl,-Bdynamic -ldl
```https://gitlab.kitware.com/sensei/sensei/-/issues/5Require adios 1.132021-02-01T19:19:57-05:00Burlen LoringRequire adios 1.13It has bug fixes we needIt has bug fixes we needhttps://gitlab.kitware.com/sensei/sensei/-/issues/11Histogram analysis segfaults if AddArray() method returns false when getting ...2021-02-01T19:19:38-05:00Andrew BauerHistogram analysis segfaults if AddArray() method returns false when getting the arrayhttps://gitlab.kitware.com/sensei/sensei/-/issues/13static linked nightly dahsboards2021-02-01T19:19:18-05:00Burlen Loringstatic linked nightly dahsboardswe need em. see #3we need em. see #3https://gitlab.kitware.com/sensei/sensei/-/issues/35Improve the web site random image feature2021-02-01T19:18:55-05:00Burlen LoringImprove the web site random image feature@patrick\-oleary @ewbethel @dcthomp
The web site random image currently does a true random walk through the image collection. This is problematic because many duplicate images are displayed. For example the first image may appear multi...@patrick\-oleary @ewbethel @dcthomp
The web site random image currently does a true random walk through the image collection. This is problematic because many duplicate images are displayed. For example the first image may appear multiple times in the first few transitions.
A simple fix would be to apply an a fixed/pre-determined cyclic order on the images and pick the starting point at random. from the randomly chosen starting point, the predetermined cyclic order would be followed. this would maximize time between repeated images, and give all the images fair air time.https://gitlab.kitware.com/sensei/sensei/-/issues/14VisIt libsim metadata requirements2021-02-01T19:18:35-05:00Burlen LoringVisIt libsim metadata requirementsLibsim needs mesh type and the number of local and global domains before accessing any data. Currently the adaptor uses the GetCompleteMesh DataAdaptor method, but that could be quite expensive for simulations that source multiple meshes...Libsim needs mesh type and the number of local and global domains before accessing any data. Currently the adaptor uses the GetCompleteMesh DataAdaptor method, but that could be quite expensive for simulations that source multiple meshes and the analysis does not make use of all of them. Currently the mesh is cached in the adaptor, but this is cumbersome and duplicative.https://gitlab.kitware.com/sensei/sensei/-/issues/15python3 compilation fails2021-02-01T19:18:06-05:00Jean M. Favrepython3 compilation failssenseiPyObject.h uses PyInt_* which apparently are gone from python3.
I read in the py3c doc that only PyLong_* remains.
I get a compile error:
/local/apps/sensei/python/senseiPyObject.h:71:25: error: ‘PyInt_FromSsize_t’ was not decla...senseiPyObject.h uses PyInt_* which apparently are gone from python3.
I read in the py3c doc that only PyLong_* remains.
I get a compile error:
/local/apps/sensei/python/senseiPyObject.h:71:25: error: ‘PyInt_FromSsize_t’ was not declared in this scopehttps://gitlab.kitware.com/sensei/sensei/-/issues/16Memory leak in VTKDataAdaptor2021-02-01T19:17:11-05:00Andrew BauerMemory leak in VTKDataAdaptorRunning the `testHistogramSerial` test with VTK_DEBUG_LEAKS enabled will show the leaks. The leak is in VTKDataAdaptor::GetMesh():
```
//----------------------------------------------------------------------------
int VTKDataAdaptor::Get...Running the `testHistogramSerial` test with VTK_DEBUG_LEAKS enabled will show the leaks. The leak is in VTKDataAdaptor::GetMesh():
```
//----------------------------------------------------------------------------
int VTKDataAdaptor::GetMesh(const std::string &meshName, bool structureOnly,
vtkDataObject *&mesh)
{
vtkDataObject *dobj = nullptr;
if (this->GetDataObject(meshName, dobj))
{
SENSEI_ERROR("Failed to get mesh \"" << meshName << "\"")
return -1;
}
if (vtkCompositeDataSet *cd = dynamic_cast<vtkCompositeDataSet*>(dobj))
{
vtkCompositeDataSet *cdo = cd->NewInstance(); ---------------------- maybe a memory leak
cdo->CopyStructure(cd);
vtkCompositeDataIterator *cdit = cd->NewIterator(); ------------------------ probably a memory leak
while (!cdit->IsDoneWithTraversal())
{
vtkDataObject *dobj = cd->GetDataSet(cdit);
vtkDataObject *dobjo = dobj->NewInstance(); ---------------------- maybe a memory leak
if (!structureOnly)
{
if (vtkDataSet *ds = dynamic_cast<vtkDataSet*>(dobj))
{
vtkDataSet *dso = static_cast<vtkDataSet*>(dobjo);
dso->CopyStructure(ds);
}
}
cdo->SetDataSet(cdit, dobjo);
cdit->GoToNextItem();
}
mesh = cdo;
return 0;
}
if (vtkDataSet *ds = dynamic_cast<vtkDataSet*>(dobj))
{
vtkDataSet *dsOut = ds->NewInstance(); ------------------- definitely a memory leak
if (!structureOnly)
dsOut->CopyStructure(ds);
mesh = dsOut;
return 0;
}
SENSEI_ERROR("Unsupoorted data object type " << dobj->GetClassName())
return -1;
}
```https://gitlab.kitware.com/sensei/sensei/-/issues/17ghost cell/node api is missing from programmable data adaptor, vtk data adapt...2021-02-01T19:16:32-05:00Burlen Loringghost cell/node api is missing from programmable data adaptor, vtk data adaptor, and othershttps://gitlab.kitware.com/sensei/sensei/-/issues/22segv in parallel3d with histogram2021-02-01T19:16:12-05:00Burlen Loringsegv in parallel3d with histogram```bash
$gdb --args ./bin/3D_Grid -g 1x1x1 -l 1x1x1 -f ../sensei_2/configs/3dgrid.xml
Global Dimensions 1X1X1: Local Dimensions 1X1X1
Total Blocks are 1X1X1
STATUS: [0][/home/sensei/sc17/software/sensei/builds/sensei_2/sensei/Configur...```bash
$gdb --args ./bin/3D_Grid -g 1x1x1 -l 1x1x1 -f ../sensei_2/configs/3dgrid.xml
Global Dimensions 1X1X1: Local Dimensions 1X1X1
Total Blocks are 1X1X1
STATUS: [0][/home/sensei/sc17/software/sensei/builds/sensei_2/sensei/ConfigurableAnalysis.cxx:198][v1.1.0-57-ga2bf4db]
STATUS: Configured histogram with 10 celldata array pressure on mesh mesh
STATUS: [0][/home/sensei/sc17/software/sensei/builds/sensei_2/sensei/ConfigurableAnalysis.cxx:198][v1.1.0-57-ga2bf4db]
STATUS: Configured histogram with 10 celldata array density on mesh mesh
STATUS: [0][/home/sensei/sc17/software/sensei/builds/sensei_2/sensei/ConfigurableAnalysis.cxx:198][v1.1.0-57-ga2bf4db]
STATUS: Configured histogram with 10 celldata array temperature on mesh mesh
Program received signal SIGSEGV, Segmentation fault.
0x000000000053c118 in sensei::VTKHistogram::Internals::operator()<double> (this=0xaa4340, array=...)
at /home/sensei/sc17/software/sensei/builds/sensei_2/sensei/VTKHistogram.cxx:115
115 ++this->Histogram[bin];
Missing separate debuginfos, use: dnf debuginfo-install libuuid-2.30-1.fc26.x86_64 python2-libs-2.7.13-11.fc26.x86_64
(gdb) where
#0 0x000000000053c118 in sensei::VTKHistogram::Internals::operator()<double> (this=0xaa4340, array=...)
at /home/sensei/sc17/software/sensei/builds/sensei_2/sensei/VTKHistogram.cxx:115
#1 0x000000000053a4ed in vtkDataArrayDispatcher<sensei::VTKHistogram::Internals, void>::Go (this=0x7fffffffccc0, lhs=0xaa3c10)
at /home/sensei/sc17/software/paraview/builds/ParaView/VTK/Common/DataModel/vtkDataArrayDispatcher.h:170
#2 0x00000000005393bd in sensei::VTKHistogram::Compute (this=0xaa3440, da=0xaa3c10, ghostArray=0x0)
at /home/sensei/sc17/software/sensei/builds/sensei_2/sensei/VTKHistogram.cxx:218
#3 0x0000000000538e70 in sensei::Histogram::Execute (this=0xaa2cb0, data=0xaa3020)
at /home/sensei/sc17/software/sensei/builds/sensei_2/sensei/Histogram.cxx:197
#4 0x000000000050145b in sensei::ConfigurableAnalysis::Execute (this=0xaa2c80, data=0xaa3020)
at /home/sensei/sc17/software/sensei/builds/sensei_2/sensei/ConfigurableAnalysis.cxx:741
#5 0x00000000004f8c62 in bridge_update (tstep=0, time=0) at /home/sensei/sc17/software/sensei/builds/sensei_2/miniapps/parallel3d/Bridge.cxx:59
#6 0x00000000004f37fa in main (argc=<optimized out>, argv=<optimized out>)
at /home/sensei/sc17/software/sensei/builds/sensei_2/miniapps/parallel3d/parallel_3D.c:162
(gdb) p bin
$12 = -2147483648
```https://gitlab.kitware.com/sensei/sensei/-/issues/33Catalyst Warning2021-02-01T19:15:38-05:00Burlen LoringCatalyst Warning@patrick\-oleary @ewbethel
Hi Patrick,
With the update to ParaView 5.6.0 I am seeing warnings every time Catalyst executes. We are passing the correct time step, ParaView warns that it only sees the previous value. However, the correc...@patrick\-oleary @ewbethel
Hi Patrick,
With the update to ParaView 5.6.0 I am seeing warnings every time Catalyst executes. We are passing the correct time step, ParaView warns that it only sees the previous value. However, the correct time value(the one we pass) is used by ParaView during the execution so that this is more of an annoyance. However, the AMReX folks use the output put streams for diagnostic and profiling data so this is a real problem for them.
It would be a big help if Kitware could help track down the problem. Is that something you can do?
```bash
STEP 22 starts ...
STEP 22 ends. TIME = 9.556420749e-15 DT = 4.343827613e-16
Walltime = 145.8195755 s; This step = 0.203643828 s; Avg. per step = 6.628162524 s
SENSEI Begin update...
Warning: In /work/SENSEI/ParaView/ParaViewCore/VTKExtensions/Core/vtkPVTrivialProducer.cxx, line 95
vtkPVTrivialProducer (0x1c35b100): Requesting time 9.12204e-15 but only 9.55642e-15 is available
SENSEI update complete (6.599138309 sec)
```https://gitlab.kitware.com/sensei/sensei/-/issues/44FLEXPATH cant send more than 1 block2021-02-01T19:15:17-05:00Burlen LoringFLEXPATH cant send more than 1 blockin many cases there are more blocks than ranks. ex AMR and Estelle's use case. she has 1 sender rank and wants to use many receiver ranks. Greg said that this is a "feature" of flexpath where they need to use a vector instead of a pointe...in many cases there are more blocks than ranks. ex AMR and Estelle's use case. she has 1 sender rank and wants to use many receiver ranks. Greg said that this is a "feature" of flexpath where they need to use a vector instead of a pointer. it may be an easy fix, and will affect adios 2 as well.
below we have 4 blocks with 1 sender and 2 receivers. this is from our regression tests, and can be used to reproduce.
**`ERROR: set_format: The Flexpath transport does not allow multiple writes using the same name in a single group, variable data_object_0/data_array_0/data is disallowed`**
```
$ testPartitionersDriver.sh "/usr/local/bin/mpiexec" "-n" "1" "2" "/Users/estelledirand/Software/Sensei/src/sensei/sensei/testing" "adios1_flexpath" "2" "2" "3" 1
Testing adios1_flexpath M=1 N=2 part_M=block part_N=block ... file=test.bp
rm -f $file ${file}_writer_info.txt
${mpiexec} ${npflag} ${nproc_write} python ${srcdir}/testPartitionersWrite.py "${srcdir}/${writer_analysis_xml}" ${nits} ${nblock_x} ${nblock_y} 16 16 -6.2832 6.2832 -6.2832 6.2832 0 6.2832
writePid=$!
[[ "${sync_mode}" == "0" ]]
[[ "${sync_mode}" == "1" ]]
echo "waiting for writer to start ${delay}"
waiting for writer to start 1
[[ -n True ]]
[[ -e "${file}_writer_info.txt" ]]
[[ ${maxDelay} -le 0 ]]
sleep ${delay}s
let maxDelay=${maxDelay}-${delay}
[[ -n True ]]
[[ -e "${file}_writer_info.txt" ]]
[[ ${maxDelay} -le 0 ]]
sleep ${delay}s
STATUS[0] : running nIts=3 with mesh parametersnblks=[2, 2] nx=[16, 16] dom=[-6.2832, 6.2832, -6.2832, 6.2832] dx=[0.3927, 0.3927] theta=[0.0, 6.2832] dt=3.1416
STATUS: [0][/Users/estelledirand/Software/Sensei/src/sensei/sensei/ConfigurableAnalysis.cxx:348][v2.1.1-103-gc44aede]
STATUS: Configured ADIOSAnalysisAdaptor "test.bp" method FLEXPATH
WARNING: [0][/Users/estelledirand/Software/Sensei/src/sensei/sensei/ADIOS1AnalysisAdaptor.cxx:96][v2.1.1-103-gc44aede]
WARNING: No subset specified. Writing all available data
let maxDelay=${maxDelay}-${delay}
[[ -n True ]]
[[ -e "${file}_writer_info.txt" ]]
break
${mpiexec} ${npflag} ${nproc_read} python ${srcdir}/testPartitionersRead.py "${srcdir}/${reader_analysis_xml}" "${srcdir}/${reader_transport_xml}"
STATUS[0] : initializing the transport layer
STATUS: [0][/Users/estelledirand/Software/Sensei/src/sensei/sensei/Partitioner.h:39][v2.1.1-103-gc44aede]
STATUS: Configured BlockPartitioner
STATUS: [0][/Users/estelledirand/Software/Sensei/src/sensei/sensei/ConfigurableInTransitDataAdaptor.cxx:143][v2.1.1-103-gc44aede]
STATUS: Configured "ADIOS1DataAdaptor
ERROR: set_format: The Flexpath transport does not allow multiple writes using the same name in a single group, variable data_object_0/data_array_0/data is disallowed
[MacEstelleD-J0539093:24003] *** Process received signal ***
[MacEstelleD-J0539093:24003] Signal: Segmentation fault: 11 (11)
[MacEstelleD-J0539093:24003] Signal code: Address not mapped (1)
[MacEstelleD-J0539093:24003] Failing at address: 0x0
```https://gitlab.kitware.com/sensei/sensei/-/issues/43disable adios1 tests that depend on pv if pv support isn't built2021-02-01T19:15:03-05:00Burlen Loringdisable adios1 tests that depend on pv if pv support isn't builtBurlen LoringBurlen Loringhttps://gitlab.kitware.com/sensei/sensei/-/issues/36make sure stuff doesn't use MPI after finalize2021-02-01T19:14:47-05:00Burlen Loringmake sure stuff doesn't use MPI after finalizeuse new end point. Test with VTKPostHoc I/Ouse new end point. Test with VTKPostHoc I/Ohttps://gitlab.kitware.com/sensei/sensei/-/issues/42update the ascent branch to sensei 3.02021-02-01T19:14:20-05:00Burlen Loringupdate the ascent branch to sensei 3.0sensei 3.0 has a new metadata model and major api changes. ascent branch is still using the 2.0 api and needs to be updated. The target branch for the merge should be `sensei_3_development`sensei 3.0 has a new metadata model and major api changes. ascent branch is still using the 2.0 api and needs to be updated. The target branch for the merge should be `sensei_3_development`DavidDavidhttps://gitlab.kitware.com/sensei/sensei/-/issues/52libsenseiCore.so is missing libvtkIOXML as dependency2021-02-01T19:13:59-05:00Jens GoebbertlibsenseiCore.so is missing libvtkIOXML as dependencyWhen building SENSEI 3.2.0 I have to add
`-DCMAKE_SHARED_LINKER_FLAGS="-L$EBROOTVTK/lib64/ -lvtkIOXML-9.0"`
to ensure the symbol
`_ZN28vtkXMLUnstructuredGridWriter3NewEv`
can be resolved.
Without this addition the library `lib...When building SENSEI 3.2.0 I have to add
`-DCMAKE_SHARED_LINKER_FLAGS="-L$EBROOTVTK/lib64/ -lvtkIOXML-9.0"`
to ensure the symbol
`_ZN28vtkXMLUnstructuredGridWriter3NewEv`
can be resolved.
Without this addition the library `libsenseiCore.so` is missing the library dependency `libvtkIOXML`.https://gitlab.kitware.com/sensei/sensei/-/issues/45Ascent make runs with AMReX2021-02-01T19:13:44-05:00Burlen LoringAscent make runs with AMReXAs a validation of the Ascent back end make some runs on cori.
repeat the runs we did with IAMR Rayleigh-Taylor 2048 cores. Use the same camera settings and color tables. Configs found below.
## IAMR input deck
[inputs.3d.rt_2048](/up...As a validation of the Ascent back end make some runs on cori.
repeat the runs we did with IAMR Rayleigh-Taylor 2048 cores. Use the same camera settings and color tables. Configs found below.
## IAMR input deck
[inputs.3d.rt_2048](/uploads/c37ef2e3efe887223dde698547a93bc7/inputs.3d.rt_2048)
## VisIt Libsim configs
[visit_rt_contour.session](/uploads/8e646fbfbca933ebe7ff000f81a93000/visit_rt_contour.session)
[rt_contour.xml](/uploads/8e6d4e1a8b1aec04f74d76a73868a754/rt_contour.xml)
![rt_contour_00500](/uploads/c7a97f8a27bdf54cb17cea1afb9b460a/rt_contour_00500.png)DavidDavidhttps://gitlab.kitware.com/sensei/sensei/-/issues/53Sensei with ParaView 5.8.x2021-02-01T19:10:54-05:00Christian WitzlerSensei with ParaView 5.8.xWhen trying to build Sensei with ParaView 5.8 the dependency to the component `ServerManagerRendering` is not found.
```
CMake Warning at CMake/catalyst.cmake:22 (find_package):
Found package configuration file:
/opt/paraview/lib...When trying to build Sensei with ParaView 5.8 the dependency to the component `ServerManagerRendering` is not found.
```
CMake Warning at CMake/catalyst.cmake:22 (find_package):
Found package configuration file:
/opt/paraview/lib64/cmake/paraview-5.8/paraview-config.cmake
but it set ParaView_FOUND to FALSE so package "ParaView" is considered to
be NOT FOUND. Reason given by package:
Could not find the ParaView package with the following required components:
ServerManagerRendering.
```
In ParaView 5.7 it is build here: https://github.com/Kitware/ParaView/blob/v5.7.0/ParaViewCore/ServerManager/Rendering/vtk.module
In ParaView 5.8 I can only find this: https://github.com/Kitware/ParaView/blob/v5.8.0/Remoting/ServerManager/vtk.module
Therefore this library seems to miss in 5.8. Is ParaView 5.8 supported?https://gitlab.kitware.com/sensei/sensei/-/issues/54Missing possibility to set some metadata in ProgrammableDataAdaptor from python2020-11-27T08:25:18-05:00Christian WitzlerMissing possibility to set some metadata in ProgrammableDataAdaptor from pythonI could not find a way to set BlockExtents (https://github.com/Kitware/sensei/blob/master/sensei/MeshMetadata.h#L182) from python. On the C side of things it is a 'std::vector<std::array<int,6>>', but creating and using https://github.co...I could not find a way to set BlockExtents (https://github.com/Kitware/sensei/blob/master/sensei/MeshMetadata.h#L182) from python. On the C side of things it is a 'std::vector<std::array<int,6>>', but creating and using https://github.com/Kitware/sensei/blob/master/python/senseiSTL.i#L8 in a list doesn't help.
Running:
```
>>> import sensei
>>> md = sensei.MeshMetadata.New()
>>> md.BlockExtents
```
Shows the type of BlockExtents is:
`<Swig Object of type 'std::vector< std::array< int,6 >,std::allocator< std::array< int,6 > > > *' at 0x2aaaf286cb10>`
Another way to set this Metadata would be using https://github.com/Kitware/sensei/blob/master/sensei/VTKUtils.h, but that can not be reached from python either?
What would be the correct way to set this metadata from python?