# Unit Testing¶

This section describes how the Steinwurf unit tests work, how to run them and how to extend them.

## Overview¶

The purpose of our unit tests is to verify that the various features work across a number of different platforms and compilers. Having an extensive unit test suite makes it easier for developers to ensure that their changes work as expected and do not cause regressions in unexpected areas of the library.

The goal is that all features in a library should have a corresponding test case. Often a test case is added while implementing a new feature. This makes it possible to check that the new functionality works as expected as early as possible.

The unit tests are implemented using the Google C++ Testing Framework (gtest) which defines a bunch of helpers that are useful when writing tests. You can find more information on gtest on their homepage.

## Running the tests¶

One of the first things you might want to try is to run the unit tests on your own machine. There are two ways to do this:

### Run the test binary manually¶

The test binary is built using the waf build scripts shipped with the library. You can read more about how you can get the source code and how to build it in the Getting Started section.

Once the code is built, the test binary will be located in a subfolder of the build folder that depends on your platform:

Linux
build/linux/test/xyz_tests
Mac OSX
build/darwin/test/xyz_tests
Windows
build/win32/test/xyz_tests.exe

If you are cross-compiling with an mkspec (as described in the Cross-compilation and Tool Options section), then the resulting binary will be located in:

mkspec
build/[mkspec]/test/xyz_tests

Note

Running the unit tests may take a long time on mobile and embedded platforms, since we test with an extensive set of parameters. For some projects, we defined the embedded profile to lower the complexity and speed up the tests:

./xyz_tests --profile=embedded


### Run the test as part of the build¶

In some cases, it is convenient to run the test binary as part of a build. This can be done by passing the following option to waf:

python waf --run_tests


When adding a new feature to a library, it is a good idea to also add a corresponding unit test. The source code for the different unit tests are placed in the test/src folder of the project.

All files with a .cpp file extension in the test/src will automatically be included in the test executable produced when building the project with waf.

In general we follow these guidelines regarding the unit tests:

1. Every class should have a corresponding unit test cpp file.
2. Remember to place the test file as described in Namespaces and directories

The purpose of this is to make it easy to find the unit test for a specific class. In some cases it makes sense to have multiple classes tested in the same file. In those cases we still make a placeholder .cpp file referring to the actual .cpp file where the test can be found. An example of this can be seen for some of the codecs, e.g. the class encoder located in src/kodo/rlnc/encoder.hpp is tested in test_coders.cpp but the place holder file still exists.

The placeholder file in this case (test/src/test_encoder.cpp) looks like the following:

 1 2 3 4 5 6 7 8 9 // Copyright Steinwurf ApS 2018. // Distributed under the "STEINWURF EVALUATION LICENSE 1.0". // See accompanying file LICENSE.rst or // http://www.steinwurf.com/licensing #include /// @file test_encoder.cpp The unit tests for the /// encoder wrapper are defined in test_coders.cpp 

Once the .cpp test file has been created, we can start to implement the unit test code. This is done with the help of the gtest framework.

### Dealing with type aliases¶

In some cases we have headers containing only type aliases such as using statements. We currently do not require that these are unit tested in isolation.

Regardless of whether a unit test is implemented or not we still leave a place holder .cpp file in the test folder.

## Example unit test¶

The Kodo library is build using the parameterized-inheritance/ mixin-layers C++ design technique. When unit testing a layer we try to isolate it as much as possible. To do this we typically introduce dummy layers with the sole purpose of satisfying the layer’s dependencies. To see this in action let’s look at one of the existing unit tests.

The storage_bytes_used layer is used when we want add functionality allowing us to keep track of how many useful bytes an encoder or decoder contains.

Note

In general the amount of data which can be encoded or decoded will be determined by the number of symbols we are coding and the size of every symbol in bytes (we call this the block size). However, in practical applications we sometimes do not have enough data to fill an entire block. In those cases we can add the storage_bytes_used layer to embed in every encoder and decoder the ability to store the number of actual data bytes.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 // Copyright Steinwurf ApS 2011. // Distributed under the "STEINWURF EVALUATION LICENSE 1.0". // See accompanying file LICENSE.rst or // http://www.steinwurf.com/licensing #pragma once #include #include namespace kodo_core { /// @ingroup storage_info_layers /// /// @brief Provides access to the number of useful bytes used out /// of the total size of the encoders or decoders storage. template class storage_bytes_used : public SuperCoder { public: /// Constructor storage_bytes_used() : m_bytes_used(0) {} /// @copydoc layer::initialize() template void initialize(Factory& the_factory) { SuperCoder::initialize(the_factory); m_bytes_used = 0; } /// @copydoc layer::set_bytes_used() void set_bytes_used(uint32_t bytes_used) { assert(bytes_used > 0); assert(bytes_used <= SuperCoder::block_size()); m_bytes_used = bytes_used; } /// @copydoc layer::bytes_used() uint32_t bytes_used() const { return m_bytes_used; } protected: /// The number of bytes used uint32_t m_bytes_used; }; } 

As seen, the layer depends on two functions being provided by the SuperCoder:

1. SuperCoder::initialize(the_factory)
2. SuperCoder::block_size()

Using our project documentation, it is possible to look up the purpose of the two undefined functions.

In this case we want to check that the state is correctly updated when calling set_bytes_used and that the state is correctly reset when calling initialize. The following unit test code was implemented in test/src/test_storage_bytes_used.cpp to test this:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 // Copyright Steinwurf ApS 2011. // Distributed under the "STEINWURF EVALUATION LICENSE 1.0". // See accompanying file LICENSE.rst or // http://www.steinwurf.com/licensing /// @file test_storage_bytes_used.cpp Unit tests for the /// storage_bytes_used class #include #include #include #include namespace kodo_core { // Put dummy layers and tests classes in an anonymous namespace // to avoid violations of ODF (one-definition-rule) in other // translation units namespace { /// Helper layer class dummy_layer { public: template void initialize(Factory& the_factory) { (void) the_factory; m_initialize(); } uint32_t block_size() const { return m_block_size(); } stub::function m_initialize; stub::function m_block_size; }; // Helper stack class dummy_stack : public storage_bytes_used { }; // Helper factory class dummy_factory { }; } } TEST(test_storage_bytes_used, api) { kodo_core::dummy_factory factory; kodo_core::dummy_stack stack; stack.m_block_size.set_return(10U); stack.initialize(factory); stack.set_bytes_used(9U); EXPECT_EQ(stack.m_initialize.calls(), 1U); EXPECT_TRUE(stack.m_block_size.expect_calls().with().to_bool()); EXPECT_EQ(stack.bytes_used(), 9U); // Initialize again and check that the state is reset stack.initialize(factory); EXPECT_EQ(stack.m_initialize.calls(), 2U); EXPECT_TRUE(stack.m_block_size.expect_calls().with().to_bool()); EXPECT_EQ(stack.bytes_used(), 0U); } 

In the above test code we use a test helper which allows us to easily add testing stubs to the unit test.

http://en.wikipedia.org/wiki/Test_stub

The library is called stub and is freely available under the BSD license:

https://github.com/steinwurf/stub

## Naming the test case¶

When we define a test using gtest we use the TEST(test_case_name, test_name) macro to define and name a test function. We follow this naming guideline:

1. The test_case_name should match the name of the .cpp file. For example, test_my_fancy_code.cpp contains test_my_fancy_code test cases. This should make it trivial to find the .cpp if a unit test fails. If a project contains multiple classes with the same name (in different subfolders), then it is recommended to modify those class names. If that is not possible, then we can add the subfolder as a prefix to the test_case_name, e.g. subfolder_test_my_fancy_code. Remember to place the test files as described in Namespaces and directories
2. The test_name is up to the developer, but should be as descriptive of the purpose of the unit test as possible.