generators.md
1 <a id="top"></a> 2 # Data Generators 3 4 > Introduced in Catch2 2.6.0. 5 6 Data generators (also known as _data driven/parametrized test cases_) 7 let you reuse the same set of assertions across different input values. 8 In Catch2, this means that they respect the ordering and nesting 9 of the `TEST_CASE` and `SECTION` macros, and their nested sections 10 are run once per each value in a generator. 11 12 This is best explained with an example: 13 ```cpp 14 TEST_CASE("Generators") { 15 auto i = GENERATE(1, 3, 5); 16 REQUIRE(is_odd(i)); 17 } 18 ``` 19 20 The "Generators" `TEST_CASE` will be entered 3 times, and the value of 21 `i` will be 1, 3, and 5 in turn. `GENERATE`s can also be used multiple 22 times at the same scope, in which case the result will be a cartesian 23 product of all elements in the generators. This means that in the snippet 24 below, the test case will be run 6 (2\*3) times. 25 26 ```cpp 27 TEST_CASE("Generators") { 28 auto i = GENERATE(1, 2); 29 auto j = GENERATE(3, 4, 5); 30 } 31 ``` 32 33 There are 2 parts to generators in Catch2, the `GENERATE` macro together 34 with the already provided generators, and the `IGenerator<T>` interface 35 that allows users to implement their own generators. 36 37 38 ## Combining `GENERATE` and `SECTION`. 39 40 `GENERATE` can be seen as an implicit `SECTION`, that goes from the place 41 `GENERATE` is used, to the end of the scope. This can be used for various 42 effects. The simplest usage is shown below, where the `SECTION` "one" 43 runs 4 (2\*2) times, and `SECTION` "two" is run 6 times (2\*3). 44 45 ```cpp 46 TEST_CASE("Generators") { 47 auto i = GENERATE(1, 2); 48 SECTION("one") { 49 auto j = GENERATE(-3, -2); 50 REQUIRE(j < i); 51 } 52 SECTION("two") { 53 auto k = GENERATE(4, 5, 6); 54 REQUIRE(i != k); 55 } 56 } 57 ``` 58 59 The specific order of the `SECTION`s will be "one", "one", "two", "two", 60 "two", "one"... 61 62 63 The fact that `GENERATE` introduces a virtual `SECTION` can also be used 64 to make a generator replay only some `SECTION`s, without having to 65 explicitly add a `SECTION`. As an example, the code below reports 3 66 assertions, because the "first" section is run once, but the "second" 67 section is run twice. 68 69 ```cpp 70 TEST_CASE("GENERATE between SECTIONs") { 71 SECTION("first") { REQUIRE(true); } 72 auto _ = GENERATE(1, 2); 73 SECTION("second") { REQUIRE(true); } 74 } 75 ``` 76 77 This can lead to surprisingly complex test flows. As an example, the test 78 below will report 14 assertions: 79 80 ```cpp 81 TEST_CASE("Complex mix of sections and generates") { 82 auto i = GENERATE(1, 2); 83 SECTION("A") { 84 SUCCEED("A"); 85 } 86 auto j = GENERATE(3, 4); 87 SECTION("B") { 88 SUCCEED("B"); 89 } 90 auto k = GENERATE(5, 6); 91 SUCCEED(); 92 } 93 ``` 94 95 > The ability to place `GENERATE` between two `SECTION`s was [introduced](https://github.com/catchorg/Catch2/issues/1938) in Catch2 2.13.0. 96 97 ## Provided generators 98 99 Catch2's provided generator functionality consists of three parts, 100 101 * `GENERATE` macro, that serves to integrate generator expression with 102 a test case, 103 * 2 fundamental generators 104 * `SingleValueGenerator<T>` -- contains only single element 105 * `FixedValuesGenerator<T>` -- contains multiple elements 106 * 5 generic generators that modify other generators 107 * `FilterGenerator<T, Predicate>` -- filters out elements from a generator 108 for which the predicate returns "false" 109 * `TakeGenerator<T>` -- takes first `n` elements from a generator 110 * `RepeatGenerator<T>` -- repeats output from a generator `n` times 111 * `MapGenerator<T, U, Func>` -- returns the result of applying `Func` 112 on elements from a different generator 113 * `ChunkGenerator<T>` -- returns chunks (inside `std::vector`) of n elements from a generator 114 * 4 specific purpose generators 115 * `RandomIntegerGenerator<Integral>` -- generates random Integrals from range 116 * `RandomFloatGenerator<Float>` -- generates random Floats from range 117 * `RangeGenerator<T>(first, last)` -- generates all values inside a `[first, last)` arithmetic range 118 * `IteratorGenerator<T>` -- copies and returns values from an iterator range 119 120 > `ChunkGenerator<T>`, `RandomIntegerGenerator<Integral>`, `RandomFloatGenerator<Float>` and `RangeGenerator<T>` were introduced in Catch2 2.7.0. 121 122 > `IteratorGenerator<T>` was introduced in Catch2 2.10.0. 123 124 The generators also have associated helper functions that infer their 125 type, making their usage much nicer. These are 126 127 * `value(T&&)` for `SingleValueGenerator<T>` 128 * `values(std::initializer_list<T>)` for `FixedValuesGenerator<T>` 129 * `table<Ts...>(std::initializer_list<std::tuple<Ts...>>)` for `FixedValuesGenerator<std::tuple<Ts...>>` 130 * `filter(predicate, GeneratorWrapper<T>&&)` for `FilterGenerator<T, Predicate>` 131 * `take(count, GeneratorWrapper<T>&&)` for `TakeGenerator<T>` 132 * `repeat(repeats, GeneratorWrapper<T>&&)` for `RepeatGenerator<T>` 133 * `map(func, GeneratorWrapper<T>&&)` for `MapGenerator<T, U, Func>` (map `U` to `T`, deduced from `Func`) 134 * `map<T>(func, GeneratorWrapper<U>&&)` for `MapGenerator<T, U, Func>` (map `U` to `T`) 135 * `chunk(chunk-size, GeneratorWrapper<T>&&)` for `ChunkGenerator<T>` 136 * `random(IntegerOrFloat a, IntegerOrFloat b)` for `RandomIntegerGenerator` or `RandomFloatGenerator` 137 * `range(Arithmetic start, Arithmetic end)` for `RangeGenerator<Arithmetic>` with a step size of `1` 138 * `range(Arithmetic start, Arithmetic end, Arithmetic step)` for `RangeGenerator<Arithmetic>` with a custom step size 139 * `from_range(InputIterator from, InputIterator to)` for `IteratorGenerator<T>` 140 * `from_range(Container const&)` for `IteratorGenerator<T>` 141 142 > `chunk()`, `random()` and both `range()` functions were introduced in Catch2 2.7.0. 143 144 > `from_range` has been introduced in Catch2 2.10.0 145 146 > `range()` for floating point numbers has been introduced in Catch2 2.11.0 147 148 And can be used as shown in the example below to create a generator 149 that returns 100 odd random number: 150 151 ```cpp 152 TEST_CASE("Generating random ints", "[example][generator]") { 153 SECTION("Deducing functions") { 154 auto i = GENERATE(take(100, filter([](int i) { return i % 2 == 1; }, random(-100, 100)))); 155 REQUIRE(i > -100); 156 REQUIRE(i < 100); 157 REQUIRE(i % 2 == 1); 158 } 159 } 160 ``` 161 162 163 Apart from registering generators with Catch2, the `GENERATE` macro has 164 one more purpose, and that is to provide simple way of generating trivial 165 generators, as seen in the first example on this page, where we used it 166 as `auto i = GENERATE(1, 2, 3);`. This usage converted each of the three 167 literals into a single `SingleValueGenerator<int>` and then placed them all in 168 a special generator that concatenates other generators. It can also be 169 used with other generators as arguments, such as `auto i = GENERATE(0, 2, 170 take(100, random(300, 3000)));`. This is useful e.g. if you know that 171 specific inputs are problematic and want to test them separately/first. 172 173 **For safety reasons, you cannot use variables inside the `GENERATE` macro. 174 This is done because the generator expression _will_ outlive the outside 175 scope and thus capturing references is dangerous. If you need to use 176 variables inside the generator expression, make sure you thought through 177 the lifetime implications and use `GENERATE_COPY` or `GENERATE_REF`.** 178 179 > `GENERATE_COPY` and `GENERATE_REF` were introduced in Catch2 2.7.1. 180 181 You can also override the inferred type by using `as<type>` as the first 182 argument to the macro. This can be useful when dealing with string literals, 183 if you want them to come out as `std::string`: 184 185 ```cpp 186 TEST_CASE("type conversion", "[generators]") { 187 auto str = GENERATE(as<std::string>{}, "a", "bb", "ccc"); 188 REQUIRE(str.size() > 0); 189 } 190 ``` 191 192 193 ### Random number generators: details 194 195 > This section applies from Catch2 3.5.0. Before that, random generators 196 > were a thin wrapper around distributions from `<random>`. 197 198 All of the `random(a, b)` generators in Catch2 currently generate uniformly 199 distributed number in closed interval \[a; b\]. This is different from 200 `std::uniform_real_distribution`, which should return numbers in interval 201 \[a; b) (but due to rounding can end up returning b anyway), but the 202 difference is intentional, so that `random(a, a)` makes sense. If there is 203 enough interest from users, we can provide API to pick any of CC, CO, OC, 204 or OO ranges. 205 206 Unlike `std::uniform_int_distribution`, Catch2's generators also support 207 various single-byte integral types, such as `char` or `bool`. 208 209 Given the same seed, the output from the integral generators is 210 reproducible across different platforms. For floating point generators, 211 we only promise reproducibility on platforms that obey the IEEE 754 212 standard, and where `float` is 4 bytes and `double` is 8 bytes. We provide 213 no guarantees for `long double`, as the internals of `long double` can 214 vary wildly across different platforms. 215 216 217 ## Generator interface 218 219 You can also implement your own generators, by deriving from the 220 `IGenerator<T>` interface: 221 222 ```cpp 223 template<typename T> 224 struct IGenerator : GeneratorUntypedBase { 225 // via GeneratorUntypedBase: 226 // Attempts to move the generator to the next element. 227 // Returns true if successful (and thus has another element that can be read) 228 virtual bool next() = 0; 229 230 // Precondition: 231 // The generator is either freshly constructed or the last call to next() returned true 232 virtual T const& get() const = 0; 233 234 // Returns user-friendly string showing the current generator element 235 // Does not have to be overridden, IGenerator provides default implementation 236 virtual std::string stringifyImpl() const; 237 }; 238 ``` 239 240 However, to be able to use your custom generator inside `GENERATE`, it 241 will need to be wrapped inside a `GeneratorWrapper<T>`. 242 `GeneratorWrapper<T>` is a value wrapper around a 243 `Catch::Detail::unique_ptr<IGenerator<T>>`. 244 245 For full example of implementing your own generator, look into Catch2's 246 examples, specifically 247 [Generators: Create your own generator](../examples/300-Gen-OwnGenerator.cpp). 248 249 250 ### Handling empty generators 251 252 The generator interface assumes that a generator always has at least one 253 element. This is not always true, e.g. if the generator depends on an external 254 datafile, the file might be missing. 255 256 There are two ways to handle this, depending on whether you want this 257 to be an error or not. 258 259 * If empty generator **is** an error, throw an exception in constructor. 260 * If empty generator **is not** an error, use the [`SKIP`](skipping-passing-failing.md#skipping-test-cases-at-runtime) in constructor. 261 262 263 264 --- 265 266 [Home](Readme.md#top)