CMake & CTest : make test doesn't build tests
DependenciesCmakeCtestDependencies Problem Overview
I'm trying CTest in CMake in order to automatically run some of my tests using make test
target. The problem is CMake does not "understand" that the test I'm willing to run has to be built since it is part of the project.
So I'm looking for a way to explicitly specify this dependency.
Dependencies Solutions
Solution 1 - Dependencies
It is arguably a bug in CMake (previously tracked here) that this doesn't work out of the box. A workaround is to do the following:
add_test(TestName ExeName)
add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND}
DEPENDS ExeName)
Then you can run make check
and it will compile and run the test. If you have several tests, then you would have to use DEPENDS exe1 exe2 exe3 ...
in the above line.
Solution 2 - Dependencies
There is actually a way to use make test
. You need to define the build of the test executable as one of the tests and then add dependencies between the tests. That is:
ADD_TEST(ctest_build_test_code
"${CMAKE_COMMAND}" --build ${CMAKE_BINARY_DIR} --target test_code)
ADD_TEST(ctest_run_test_code test_code)
SET_TESTS_PROPERTIES(ctest_run_test_code
PROPERTIES DEPENDS ctest_build_test_code)
Solution 3 - Dependencies
I use a variant of richq's answer. In the top-level CMakeLists.txt
, I add a custom target, build_and_test
, for building and running all tests:
find_package(GTest)
if (GTEST_FOUND)
enable_testing()
add_custom_target(build_and_test ${CMAKE_CTEST_COMMAND} -V)
add_subdirectory(test)
endif()
In the various sub-project CMakeLists.txt
files under test/
, I add each test executable as a dependency of build_and_test
:
include_directories(${CMAKE_SOURCE_DIR}/src/proj1)
include_directories(${GTEST_INCLUDE_DIRS})
add_executable(proj1_test proj1_test.cpp)
target_link_libraries(proj1_test ${GTEST_BOTH_LIBRARIES} pthread)
add_test(proj1_test proj1_test)
add_dependencies(build_and_test proj1_test)
With this approach, I just need to make build_and_test
instead of make test
(or make all test
), and it has the benefit of only building test code (and its dependencies). It's a shame I can't use the target name test
. In my case, it's not so bad because I have a top-level script that does out-of-tree debug and release (and cross-compiled) builds by calling cmake
and then make
, and it translates test
into build_and_test
.
Obviously, the GTest stuff isn't required. I just happen to use/like Google Test, and wanted to share a complete example of using it with CMake/CTest. IMHO, this approach also has the benefit of allowing me to use ctest -V
, which shows the Google Test output while the tests run:
1: Running main() from gtest_main.cc
1: [==========] Running 1 test from 1 test case.
1: [----------] Global test environment set-up.
1: [----------] 1 test from proj1
1: [ RUN ] proj1.dummy
1: [ OK ] proj1.dummy (0 ms)
1: [----------] 1 test from proj1 (1 ms total)
1:
1: [----------] Global test environment tear-down
1: [==========] 1 test from 1 test case ran. (1 ms total)
1: [ PASSED ] 1 test.
1/2 Test #1: proj1_test ....................... Passed 0.03 sec
Solution 4 - Dependencies
If you are using CMake >= 3.7, then the recommended approach is to use fixtures:
add_executable(test test.cpp)
add_test(test_build
"${CMAKE_COMMAND}"
--build "${CMAKE_BINARY_DIR}"
--config "$<CONFIG>"
--target test
)
set_tests_properties(test_build PROPERTIES FIXTURES_SETUP test_fixture)
add_test(test test)
set_tests_properties(test PROPERTIES FIXTURES_REQUIRED test_fixture)
This does the following:
- Adds a
test
executable target built fromtest.cpp
- Adds a
test_build
"test" that runs Cmake to build targettest
- Marks the
test_build
test to be a setup task of fixturetest_fixture
- Add a
test
test that just runs thetest
executable - Marks the
test
test to need fixturetest_fixture
.
So, every time test test
is to be run, it first runs test test_build
, which builds the necessary executable.
Solution 5 - Dependencies
If you are trying to emulate make check
, you may find this wiki entry usefull :
http://www.cmake.org/Wiki/CMakeEmulateMakeCheck
I have just checked that is does what it says with success (CMake 2.8.10).
Solution 6 - Dependencies
Save yourself the headache:
make all test
Works out of the box for me and will build dependencies before running the test. Given how simple this is, it almost makes the native make test
functionality convenient because it gives you the option of running the last compiling tests even if your code is broken.
Solution 7 - Dependencies
This is what I hammered out and have been using:
set(${PROJECT_NAME}_TESTS a b c)
enable_testing()
add_custom_target(all_tests)
foreach(test ${${PROJECT_NAME}_TESTS})
add_executable(${test} EXCLUDE_FROM_ALL ${test}.cc)
add_test(NAME ${test} COMMAND $<TARGET_FILE:${test}>)
add_dependencies(all_tests ${test})
endforeach(test)
build_command(CTEST_CUSTOM_PRE_TEST TARGET all_tests)
string(CONFIGURE \"@CTEST_CUSTOM_PRE_TEST@\" CTEST_CUSTOM_PRE_TEST_QUOTED ESCAPE_QUOTES)
file(WRITE "${CMAKE_BINARY_DIR}/CTestCustom.cmake" "set(CTEST_CUSTOM_PRE_TEST ${CTEST_CUSTOM_PRE_TEST_QUOTED})" "\n")
YMMV
Solution 8 - Dependencies
For CMake 3.10 or later, another option is to use the TEST_INCLUDE_FILES
directory property to set up a script that triggers a build before a test is run. In your outermost CMakeLists.txt
add the following code:
set_property(DIRECTORY APPEND
PROPERTY TEST_INCLUDE_FILES "${CMAKE_CURRENT_BINARY_DIR}/BuildTestTarget.cmake")
file(WRITE "${CMAKE_CURRENT_BINARY_DIR}/BuildTestTarget.cmake"
"execute_process(COMMAND \"${CMAKE_COMMAND}\""
" --build \"${CMAKE_BINARY_DIR}\""
" --config \"\$ENV{CMAKE_CONFIG_TYPE}\")")
The actual test configuration is passed through to the build via the environment variable CMAKE_CONFIG_TYPE
. Optionally you can add a --target
option to only build targets required by the test.
Solution 9 - Dependencies
Derrick's answer, simplified and commented:
# It is impossible to make target "test" depend on "all":
# https://gitlab.kitware.com/cmake/cmake/-/issues/8774
# Set a magic variable in a magic file that tells ctest
# to invoke the generator once before running the tests:
file(WRITE "${CMAKE_BINARY_DIR}/CTestCustom.cmake"
"set(CTEST_CUSTOM_PRE_TEST ${CMAKE_MAKE_PROGRAM})\n"
)
It is not perfectly correct, as it does not solve the concurrency problem of running ninja all test
, in case anyone does that. On the contrary, because now, you have two ninja processes.
(Ftr, I also shared this solution here.)
Solution 10 - Dependencies
All above answers are perfect. But actually CMake use CTest as its testing tools, so the standard method (I think it is) to do the mission is:
enable_testing ()
add_test (TestName TestCommand)
add_test (TestName2 AnotherTestCommand)
Then run cmake and make to build the targets. After that, you can either run make test, or just run
ctest
you will get the result. This is tested under CMake 2.8 .
Check details at: http://cmake.org/Wiki/CMake/Testing_With_CTest#Simple_Testing
Solution 11 - Dependencies
All the answers are good, but they imply a breach of tradition to run a test by command make test
. I've done this trick:
add_test(NAME <mytest>
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
COMMAND sh -c "make <mytarget>; $<TARGET_FILE:<mytarget>>")
This means that the test consists of building (optionally) and running of executable target.