If our ``find_library`` step hasn't used a particular link directory at
all, it's best to retain it in order to prevent issues from secondary
dependencies not being found.