CMake/CTest - execute processes in parallel as a test

in my team we develop a library which coordinates multiple applications over some network back-end. The building is handled by CMake 3.10.4 and we run our tests with CTest. We currently support only Linux, but Windows and MacOS support is on the horizon.

We also provide example applications which we configure and build as tests in CTests.

How can we:

1. run 2 example programs with CTest as a single test in parallel?
2. keep the implementation as system-independent as possible?
3. capture both STDOUT separately (interleaved is fine too)?

What we tried

We created a CMake script which is called by CTest. It executes the following using execute_process:

# DUMMY_A, DUMMY_B are the dummy application
execute_process(
  COMMAND ${DUMMY_A} ${DUMMY_CONFIG} SolverOne MeshOne
  COMMAND ${DUMMY_B} ${DUMMY_CONFIG} SolverTwo MeshTwo
  WORKING_DIRECTORY ${DUMMY_RUN_DIR}
  RESULTS_VARIABLE DUMMY_RESULTS
)

This does not work reliably as DUMMY_B may return prior to DUMMY_A. In this case, DUMMY_A still writes to STDOUT which leads to a SIGPIPE (broken pipe) and thus a failing test.

Further options we considered:

• Using a shellscript to run both programs in parallel, which is system dependent.
• Using a python script to run one program as subprocess, which obviously requires python.
• Allowing the tests to fail with SIGPIPE which we can check via RESULTS_VARIABLE. This could hide actual failing tests though.