sensei issueshttps://gitlab.kitware.com/sensei/sensei/-/issues2021-02-01T19:16:32-05:00https://gitlab.kitware.com/sensei/sensei/-/issues/17ghost cell/node api is missing from programmable data adaptor, vtk data adapt...2021-02-01T19:16:32-05:00Burlen Loringghost cell/node api is missing from programmable data adaptor, vtk data adaptor, and othershttps://gitlab.kitware.com/sensei/sensei/-/issues/16Memory leak in VTKDataAdaptor2021-02-01T19:17:11-05:00Andrew BauerMemory leak in VTKDataAdaptorRunning the `testHistogramSerial` test with VTK_DEBUG_LEAKS enabled will show the leaks. The leak is in VTKDataAdaptor::GetMesh():
```
//----------------------------------------------------------------------------
int VTKDataAdaptor::Get...Running the `testHistogramSerial` test with VTK_DEBUG_LEAKS enabled will show the leaks. The leak is in VTKDataAdaptor::GetMesh():
```
//----------------------------------------------------------------------------
int VTKDataAdaptor::GetMesh(const std::string &meshName, bool structureOnly,
vtkDataObject *&mesh)
{
vtkDataObject *dobj = nullptr;
if (this->GetDataObject(meshName, dobj))
{
SENSEI_ERROR("Failed to get mesh \"" << meshName << "\"")
return -1;
}
if (vtkCompositeDataSet *cd = dynamic_cast<vtkCompositeDataSet*>(dobj))
{
vtkCompositeDataSet *cdo = cd->NewInstance(); ---------------------- maybe a memory leak
cdo->CopyStructure(cd);
vtkCompositeDataIterator *cdit = cd->NewIterator(); ------------------------ probably a memory leak
while (!cdit->IsDoneWithTraversal())
{
vtkDataObject *dobj = cd->GetDataSet(cdit);
vtkDataObject *dobjo = dobj->NewInstance(); ---------------------- maybe a memory leak
if (!structureOnly)
{
if (vtkDataSet *ds = dynamic_cast<vtkDataSet*>(dobj))
{
vtkDataSet *dso = static_cast<vtkDataSet*>(dobjo);
dso->CopyStructure(ds);
}
}
cdo->SetDataSet(cdit, dobjo);
cdit->GoToNextItem();
}
mesh = cdo;
return 0;
}
if (vtkDataSet *ds = dynamic_cast<vtkDataSet*>(dobj))
{
vtkDataSet *dsOut = ds->NewInstance(); ------------------- definitely a memory leak
if (!structureOnly)
dsOut->CopyStructure(ds);
mesh = dsOut;
return 0;
}
SENSEI_ERROR("Unsupoorted data object type " << dobj->GetClassName())
return -1;
}
```https://gitlab.kitware.com/sensei/sensei/-/issues/15python3 compilation fails2021-02-01T19:18:06-05:00Jean M. Favrepython3 compilation failssenseiPyObject.h uses PyInt_* which apparently are gone from python3.
I read in the py3c doc that only PyLong_* remains.
I get a compile error:
/local/apps/sensei/python/senseiPyObject.h:71:25: error: ‘PyInt_FromSsize_t’ was not decla...senseiPyObject.h uses PyInt_* which apparently are gone from python3.
I read in the py3c doc that only PyLong_* remains.
I get a compile error:
/local/apps/sensei/python/senseiPyObject.h:71:25: error: ‘PyInt_FromSsize_t’ was not declared in this scopehttps://gitlab.kitware.com/sensei/sensei/-/issues/14VisIt libsim metadata requirements2021-02-01T19:18:35-05:00Burlen LoringVisIt libsim metadata requirementsLibsim needs mesh type and the number of local and global domains before accessing any data. Currently the adaptor uses the GetCompleteMesh DataAdaptor method, but that could be quite expensive for simulations that source multiple meshes...Libsim needs mesh type and the number of local and global domains before accessing any data. Currently the adaptor uses the GetCompleteMesh DataAdaptor method, but that could be quite expensive for simulations that source multiple meshes and the analysis does not make use of all of them. Currently the mesh is cached in the adaptor, but this is cumbersome and duplicative.https://gitlab.kitware.com/sensei/sensei/-/issues/13static linked nightly dahsboards2021-02-01T19:19:18-05:00Burlen Loringstatic linked nightly dahsboardswe need em. see #3we need em. see #3https://gitlab.kitware.com/sensei/sensei/-/issues/12FindMPI4PY.cmake and FindNUMPY.cmake not python3 friendly2018-01-29T13:53:25-05:00Jean M. FavreFindMPI4PY.cmake and FindNUMPY.cmake not python3 friendlybuilding sensei with python3 fails to find numpy and mpi4pybuilding sensei with python3 fails to find numpy and mpi4pyhttps://gitlab.kitware.com/sensei/sensei/-/issues/11Histogram analysis segfaults if AddArray() method returns false when getting ...2021-02-01T19:19:38-05:00Andrew BauerHistogram analysis segfaults if AddArray() method returns false when getting the arrayhttps://gitlab.kitware.com/sensei/sensei/-/issues/10DataAdaptor API has no way of request specific meshes2018-04-16T18:19:28-04:00Burlen LoringDataAdaptor API has no way of request specific meshesWhen a simulation uses multiple meshes it is impossible to know which one is needed.
An motivating example is the Warp LPA simulation code, which computes the solution of a number of different particle species and has a Cartesian or blo...When a simulation uses multiple meshes it is impossible to know which one is needed.
An motivating example is the Warp LPA simulation code, which computes the solution of a number of different particle species and has a Cartesian or block structured AMR mesh for the solution of megnetic and electric fields. Particle species can be molecules or sub atomic particles such as porotons and electrons. There can be multiple species of the same sub atomic particle type, for instance electrons are kept in distict species identifying which molecules/atoms they derived from. The number of species present in given simulation is problem specific.
**Existing DataAdaptor API:**
```c++
virtual vtkDataObject* GetMesh(bool structure_only=false) = 0;
virtual bool AddArray(vtkDataObject* mesh, int association, const std::string& arrayname) = 0;
virtual unsigned int GetNumberOfArrays(int association) = 0;
virtual std::string GetArrayName(int association, unsigned int index) = 0;
virtual void ReleaseData() = 0;
```
Note that in AddArray we pass a VTK pointer in which could potentially be used as a key to identify which mesh data is being requested from. However, this might be highly inconvenient and potentially problematic in languages other than C++.
**Option 1:**
```c++
// Discover the number of meshes a simulation can provide
virtual int GetNumberOfMeshes(int &numMeshes) = 0;
// Discover the names of the meshes
virtual int GetMeshName(int id, const std::string &meshName) = 0;
virtual int GetMesh(const std::string &meshName, bool structure_only,
vtkDataObject *&mesh) = 0;
virtual int AddArray(vtkDataObject* mesh, const std::string &meshName,
int association, const std::string& arrayname) = 0;
virtual int GetNumberOfArrays(const std::string &meshName, int association,
unsigned int &numberOfArrays) = 0;
virtual int GetArrayName(const std::string &meshName, int association,
unsigned int index, std::string &arrayName) = 0;
virtual int ReleaseData(const std::string &meshName) = 0;
virtual int ReleaseData() = 0;
```
`meshName` identifies the mesh where necessary. In all API an `int` is returned, zero indicates success, non zero an error (see #9).
https://gitlab.kitware.com/sensei/sensei/-/issues/9DataAadatpor API has no way to indicate method failed2018-07-20T16:59:19-04:00Burlen LoringDataAadatpor API has no way to indicate method failedIf the following methods fail there is no reliable way to let the caller know that his call has failed. Note that in the case of GetMesh it is valid to return a `nullptr`, for instance if the analysis is run on more cores than the simula...If the following methods fail there is no reliable way to let the caller know that his call has failed. Note that in the case of GetMesh it is valid to return a `nullptr`, for instance if the analysis is run on more cores than the simulation than some number of processes will return `nullptr`.
```C++
virtual vtkDataObject* GetMesh(bool structure_only=false) = 0;
virtual std::string GetArrayName(int association, unsigned int index) = 0;
```
How is the best way to indicate an error for this call? Could an adaptor not provide arrays? in that case 0 would not be an error. A negative return could indicate an error if the return type was signed.
```C++
virtual unsigned int GetNumberOfArrays(int association) = 0;
```https://gitlab.kitware.com/sensei/sensei/-/issues/7VTKPosthocIO add arrays2018-05-09T19:58:03-04:00Burlen LoringVTKPosthocIO add arraysUse SENSEI's AddArray API, currently expects them to be presentUse SENSEI's AddArray API, currently expects them to be presenthttps://gitlab.kitware.com/sensei/sensei/-/issues/6Handle non-composite data in VTKPosthocIO2018-04-16T18:17:29-04:00Burlen LoringHandle non-composite data in VTKPosthocIOHandle non-composite data in VTKPosthocIOHandle non-composite data in VTKPosthocIOhttps://gitlab.kitware.com/sensei/sensei/-/issues/5Require adios 1.132021-02-01T19:19:57-05:00Burlen LoringRequire adios 1.13It has bug fixes we needIt has bug fixes we needhttps://gitlab.kitware.com/sensei/sensei/-/issues/3libsim build issue2021-02-01T19:21:16-05:00Burlen Loringlibsim build issuerecall the error seen during the SENSEI tutorial at SC17:
```
c++ [options removed] -o ../../bin/3D_Grid ...[stuff removed] /local/apps/VisIt/2.13/current/linux-x86_64/libsim/V2/lib/libsimV2.a
/usr/bin/ld: /local/apps/VisIt/2.13/curre...recall the error seen during the SENSEI tutorial at SC17:
```
c++ [options removed] -o ../../bin/3D_Grid ...[stuff removed] /local/apps/VisIt/2.13/current/linux-x86_64/libsim/V2/lib/libsimV2.a
/usr/bin/ld: /local/apps/VisIt/2.13/current/linux-x86_64/libsim/V2/lib/libsimV2.a(VisItControlInterface_V2.c.o): undefined reference to symbol 'dlsym@@GLIBC_2.2.5'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libdl.so: error adding symbols: DSO missing from command line
```
I solved it by looking at how I compiled my own libsim demos. To fix it, I added:
```
-Wl,-Bstatic /local/apps/VisIt/2.13/current/linux-x86_64/libsim/V2/lib/libsimV2.a -Wl,-Bdynamic -ldl
```https://gitlab.kitware.com/sensei/sensei/-/issues/2clang warnings in osciallator miniapp2017-11-08T19:27:12-05:00Burlen Loringclang warnings in osciallator miniappclang reports use of static member variable that has not been initialized. here is a prototypical instance
```
In file included from /Users/bloring/sensei/miniapps/oscillators/oscillator.cpp:4:
/Users/bloring/sensei/miniapps/oscillators...clang reports use of static member variable that has not been initialized. here is a prototypical instance
```
In file included from /Users/bloring/sensei/miniapps/oscillators/oscillator.cpp:4:
/Users/bloring/sensei/miniapps/oscillators/include/format.h:589:25: warning: instantiation of variable 'fmt::internal::BasicData<void>::POWERS_OF_10_64' required here, but no definition is available [-Wundefined-var-template]
return t - (n < Data::POWERS_OF_10_64[t]) + 1;
^
/Users/bloring/sensei/miniapps/oscillators/include/format.h:568:25: note: forward declaration of template entity is here
static const uint64_t POWERS_OF_10_64[];
^
/Users/bloring/sensei/miniapps/oscillators/include/format.h:589:25: note: add an explicit instantiation declaration to suppress this warning if 'fmt::internal::BasicData<void>::POWERS_OF_10_64' is explicitly instantiated in another translation unit
return t - (n < Data::POWERS_OF_10_64[t]) + 1;
^
```
here is the class in question
```C++
563 // Static data is placed in this class template to allow header-only
564 // configuration.
565 template <typename T = void>
566 struct BasicData {
567 static const uint32_t POWERS_OF_10_32[];
568 static const uint64_t POWERS_OF_10_64[];
569 static const char DIGITS[];
570 };
```https://gitlab.kitware.com/sensei/sensei/-/issues/1Oscilators miniapp DIY deadlock2017-10-10T13:18:50-04:00Burlen LoringOscilators miniapp DIY deadlockDIY enters a deadlock when the miniapp is run with more MPI ranks than blocks. I was using this as a test case to be sure the ADIOS adaptors can work under such conditions. Commands to reproduce below, followed by backtrace.
Note that t...DIY enters a deadlock when the miniapp is run with more MPI ranks than blocks. I was using this as a test case to be sure the ADIOS adaptors can work under such conditions. Commands to reproduce below, followed by backtrace.
Note that the backtrace below shows that this issue is occurring in the destructor of the Autocorelation class. It is generally bad form to do anything but release resources in a destructor, and I suspect that this is the issue rather than a problem in DIY.
```xml
<sensei>
<analysis type="autocorrelation" array="data" association="cell"
window="10" k-max="3" enabled="1" />
</sensei>
```
```bash
mpiexec -np 2 ./bin/oscillator -b 1 -t 1 -f ./config.xml ../sensei/miniapps/oscillators/inputs/sample.osc
```
```C++
(gdb) bt
#0 0x00007fffc60c3b09 in mca_btl_vader_component_progress () at btl_vader_component.c:688
#1 0x00007fffd2678dea in opal_progress () at runtime/opal_progress.c:187
#2 0x00007fffd72862d5 in opal_condition_wait (c=<optimized out>, m=<optimized out>) at ../opal/threads/condition.h:78
#3 ompi_request_default_wait_all (count=2, requests=0x7fffffffba60, statuses=0x0) at request/req_wait.c:281
#4 0x00007fffc4e147a6 in ompi_coll_tuned_allreduce_intra_recursivedoubling (sbuf=<optimized out>, rbuf=0xd25650, count=10,
dtype=0x7fffd750d060 <ompi_mpi_float>, op=0x7fffd751db80 <ompi_mpi_op_sum>, comm=<optimized out>, module=0xc1d610) at coll_tuned_allreduce.c:245
#5 0x00007fffd7296efe in PMPI_Allreduce (sendbuf=0xd25860, recvbuf=0xd25650, count=10, e=0x7fffd750d060 <ompi_mpi_float>,
op=0x7fffd751db80 <ompi_mpi_op_sum>, comm=0x7fffd7514b20 <ompi_mpi_comm_world>) at pallreduce.c:109
#6 0x000000000052cff0 in diy::mpi::Collectives<float, add_vectors<float> >::all_reduce (comm=..., in=std::vector of length 10, capacity 10 = {...},
out=std::vector of length 10, capacity 10 = {...}) at /work/SENSEI/sensei/utils/diy/mpi/collectives.hpp:179
#7 0x000000000052cae6 in diy::mpi::all_reduce<float, add_vectors<float> > (comm=..., in=std::vector of length 10, capacity 10 = {...},
out=std::vector of length 10, capacity 10 = {...}, op=...) at /work/SENSEI/sensei/utils/diy/mpi/collectives.hpp:292
#8 0x000000000052c509 in diy::detail::AllReduceOp<std::vector<float, std::allocator<float> >, add_vectors<float> >::global (this=0x7fffb4000910, comm=...)
at /work/SENSEI/sensei/utils/diy/detail/collectives.hpp:26
#9 0x000000000051d8c0 in diy::Master::Collective::global (this=0x7fffb40009a0, c=...) at /work/SENSEI/sensei/utils/diy/master.hpp:350
#10 0x000000000051ed26 in diy::Master::process_collectives (this=0xc21ec0) at /work/SENSEI/sensei/utils/diy/master.hpp:1004
#11 0x000000000051eab8 in diy::Master::flush (this=0xc21ec0) at /work/SENSEI/sensei/utils/diy/master.hpp:974
#12 0x000000000051db31 in diy::Master::exchange (this=0xc21ec0) at /work/SENSEI/sensei/utils/diy/master.hpp:788
#13 0x0000000000566882 in sensei::Autocorrelation::PrintResults (this=0xc1d580, k_max=3) at /work/SENSEI/sensei/sensei/Autocorrelation.cxx:313
#14 0x00000000005654c6 in sensei::Autocorrelation::~Autocorrelation (this=0xc1d580, __in_chrg=<optimized out>)
at /work/SENSEI/sensei/sensei/Autocorrelation.cxx:210
#15 0x0000000000565524 in sensei::Autocorrelation::~Autocorrelation (this=0xc1d580, __in_chrg=<optimized out>)
at /work/SENSEI/sensei/sensei/Autocorrelation.cxx:212
#16 0x00007fffd8342820 in vtkObjectBase::UnRegisterInternal (this=0xc1d580, check=0) at /work/SENSEI/ParaView/VTK/Common/Core/vtkObjectBase.cxx:240
#17 0x00007fffd83426e6 in vtkObjectBase::UnRegister (this=0xc1d580, o=0x0) at /work/SENSEI/ParaView/VTK/Common/Core/vtkObjectBase.cxx:197
#18 0x00007fffd83796fb in vtkSmartPointerBase::~vtkSmartPointerBase (this=0xc1eb70, __in_chrg=<optimized out>)
at /work/SENSEI/ParaView/VTK/Common/Core/vtkSmartPointerBase.cxx:62
#19 0x0000000000549338 in vtkSmartPointer<sensei::AnalysisAdaptor>::~vtkSmartPointer (this=0xc1eb70, __in_chrg=<optimized out>)
at /work/SENSEI/ParaView/VTK/Common/Core/vtkSmartPointer.h:29
#20 0x000000000054a9e7 in std::_Destroy<vtkSmartPointer<sensei::AnalysisAdaptor> > (__pointer=0xc1eb70) at /usr/include/c++/5.3.1/bits/stl_construct.h:93
#21 0x000000000054a7d9 in std::_Destroy_aux<false>::__destroy<vtkSmartPointer<sensei::AnalysisAdaptor>*> (__first=0xc1eb70, __last=0xc1eb78)
at /usr/include/c++/5.3.1/bits/stl_construct.h:103
#22 0x000000000054a3f2 in std::_Destroy<vtkSmartPointer<sensei::AnalysisAdaptor>*> (__first=0xc1eb70, __last=0xc1eb78)
at /usr/include/c++/5.3.1/bits/stl_construct.h:126
#23 0x0000000000549e3f in std::_Destroy<vtkSmartPointer<sensei::AnalysisAdaptor>*, vtkSmartPointer<sensei::AnalysisAdaptor> > (__first=0xc1eb70,
__last=0xc1eb78) at /usr/include/c++/5.3.1/bits/stl_construct.h:151
#24 0x0000000000549877 in std::vector<vtkSmartPointer<sensei::AnalysisAdaptor>, std::allocator<vtkSmartPointer<sensei::AnalysisAdaptor> > >::~vector (
this=0xc21e90, __in_chrg=<optimized out>) at /usr/include/c++/5.3.1/bits/stl_vector.h:424
#25 0x00000000005493f6 in sensei::ConfigurableAnalysis::InternalsType::~InternalsType (this=0xc21e90, __in_chrg=<optimized out>)
at /work/SENSEI/sensei/sensei/ConfigurableAnalysis.cxx:99
#26 0x0000000000548728 in sensei::ConfigurableAnalysis::~ConfigurableAnalysis (this=0xc21e60, __in_chrg=<optimized out>)
at /work/SENSEI/sensei/sensei/ConfigurableAnalysis.cxx:501
#27 0x0000000000548770 in sensei::ConfigurableAnalysis::~ConfigurableAnalysis (this=0xc21e60, __in_chrg=<optimized out>)
at /work/SENSEI/sensei/sensei/ConfigurableAnalysis.cxx:502
#28 0x00007fffd8342820 in vtkObjectBase::UnRegisterInternal (this=0xc21e60, check=0) at /work/SENSEI/ParaView/VTK/Common/Core/vtkObjectBase.cxx:240
#29 0x00007fffd83426e6 in vtkObjectBase::UnRegister (this=0xc21e60, o=0x0) at /work/SENSEI/ParaView/VTK/Common/Core/vtkObjectBase.cxx:197
#30 0x00007fffd83796fb in vtkSmartPointerBase::~vtkSmartPointerBase (this=0x7fffffffc210, __in_chrg=<optimized out>)
at /work/SENSEI/ParaView/VTK/Common/Core/vtkSmartPointerBase.cxx:62
#31 0x00007fffd8379741 in vtkSmartPointerBase::operator= (this=0x8ceb58 <bridge::GlobalAnalysisAdaptor>, r=0x0)
at /work/SENSEI/ParaView/VTK/Common/Core/vtkSmartPointerBase.cxx:75
#32 0x00000000005313cd in vtkSmartPointer<sensei::ConfigurableAnalysis>::operator= (this=0x8ceb58 <bridge::GlobalAnalysisAdaptor>, r=0x0)
at /work/SENSEI/ParaView/VTK/Common/Core/vtkSmartPointer.h:58
#33 0x00000000005310c4 in bridge::finalize (k_max=3, nblocks=1) at /work/SENSEI/sensei/miniapps/oscillators/bridge.cpp:76
#34 0x00000000004eadeb in main (argc=8, argv=0x7fffffffd518) at /work/SENSEI/sensei/miniapps/oscillators/oscillator.cpp:267
```