Boolean Intersect returns incorrect results

See below image. The blue and yellow are my two groups of interest, the pink is the output from the intersection. White faces are faces that were incorrectly left out, and some faces such as the isolated pink triangle are incorrectly included.