WIP: Thread local variable for thread safety
The previous effort MR !2548 (merged) to make filters thread-safe for parallel processing of PartitionedDataSet
is flawed. It is pervasively invasive. Filters without mutable states are thread-safe by definition. However, they need to declare their thread safeness and implement the Clone
method. The Clone
method not only need to duplicate mutable states but also immutable states. Such a design impose unnecessary overhead for all filters, both in terms of extra code and runtime cost, since the filters need to be copied numThread
times.
The proper solution should only impact filters that are not thread-safe by default and duplicate only mutable states when necessary. This design uses C++11 feature of thread local variable for those mutable states. Each thread will have a local copy of the mutable state and the execution only mutates its own local copy, thus making the filter thread-safe.
This MR address #601 (closed) on thread safety issue for the new filter design. A follow up MR will
- make
CanThread == true
by default - update to all currently thread unsafe filters to adopt this solution
- make
DoExecute
andDoMapField
const-qualified and marking mutable states withmutable
keyword.