Skip to content

ProcessUNIX.c: fix select(), FD_SETSIZE on macOS

Christopher Chavez requested to merge chrstphrchvz/kwsys:fix-select-macos into master

On macOS (reportedly for 10.5 and later), if _DARWIN_UNLIMITED_SELECT is not defined, then the select() call in kwsysProcessWaitForPipe() fails with EINVAL when (max + 1) > 1024. And, by default, FD_SETSIZE is 1024; passing file descriptors not less than FD_SETSIZE to FD_CLR(), FD_SET(), and FD_ISSET() causes out-of-bounds accesses, at least for an fd_set allocated on the heap (as is the case for cp->PipeSet, allocated by kwsysProcess_New(); one observed effect of this is corruption of the Invalid argument error message copied from strerror(errno) into cp->ErrorMessage, since cp->ErrorMessage is located shortly after cp->PipeSet). Defining _DARWIN_UNLIMITED_SELECT and defining FD_SETSIZE to OPEN_MAX (10240) avoids these issues.

Issue: cmake/cmake#23156 (closed)

Edited by Brad King

Merge request reports