FindQt4: Revert "Set PLUGINS and IMPORTS dir even if empty"
The change in !1503 (merged) regressed existing builds on some machines. Revert it for CMake 3.11 pending further investigation.
Fixes: #17814 (closed), #17817 (closed)
Issue: #14155 (closed), #17492