How to use FsCheck to generate random numbers as input for property-based testing

Question

I thought it was time to try FsCheck, but it turned out to be tougher than I thought. There is a lot of documentation on Arb , generators, etc., but there seems to be no indication on how to apply this knowledge. Or I just don't get it.

What can complicate this is that the connection between tests, properties, generators, arbitrariness, compression, and, in my case, randomness (some tests automatically generate random data, others not) is not clear to me. I don't have a Haskell background, so that doesn't help either.

Now for the question: how can I generate random integers?

My test case can be explained using the multiplication properties of, say, distributivity:

 static member ``Multiplication is distributive`` (x: int64) yz = let res1 = x * (y + z) let res2 = x * y + x * z res1 = res2 // run it: [<Test>] static member FsCheckAsUnitTest() = Check.One({ Config.VerboseThrowOnFailure with MaxTest = 1000 }, ``Multiplication is distributive``) 

When I run this using Check.Verbose or NUnit integration, I get test sequences, for example:

 0: (-1L, -1L, -1L) 1: (-1L, -1L, 0L) 2: (-1L, -1L, -1L) 3: (-1L, -1L, -1L) 4: (-1L, 0L, -1L) 5: (1L, 0L, 2L) 6: (-2L, 0L, -1L) 7: (-2L, -1L, -1L) 8: (1L, 1L, -2L) 9: (-2L, 2L, -2L) 

After 1000 tests, he did not receive more than 100L . Somehow, I suggested that this “automatically” selects random numbers evenly distributed throughout the int64 range, at least the way I interpreted the documentation.

Since this is not the case, I started experimenting and came up with stupid solutions like the following in order to get higher numbers:

 type Generators = static member arbMyRecord = Arb.generate<int64> |> Gen.where ((<) 1000L) |> Gen.three |> Arb.fromGen 

But this is becoming incredibly slow and clearly not suitable. I am sure there should be a simple solution that I am missing. I tried with Gen.choose(Int64.MinValue, Int64.MaxValue) , but this only supports ints, not longs (but even with just ints I could not get it to work).

In the end, I need a solution that works for all primitive numeric data types, including their maxes and mins, their zeros and ones, and some random selection from what's inside.

Answer 1

As explained in this other FsCheck question , the default configurations for most Check functions have EndSize = 100 . You can increase this number, but you can also, as you think, use Gen.choose .

However, the int generator intentionally behaves well . For example, it does not include Int32.MinValue and Int32.MaxValue , as this can lead to overflows.

FsCheck, however, also comes with generators that give you uniform distributions across their entire range: Arb.Default.DoNotSizeInt16 , Arb.Default.DoNotSizeUInt64 , etc.

For floating point Arb.Default.Float32 , there is Arb.Default.Float32 , which, according to its documentation, generates "arbitrary floats", NaN, NegativeInfinity, PositiveInfinity, Maxvalue, MinValue, Epsilon is included quite often. "

There is no single API for "just" any number, since F # has no types (this is what you could express in Haskell).

Also, I'm not sure if your typical unit testing module will be able to run general tests, but at least with xUnit.net you can use this trick to run typically typed tests .

---

In particular, you can write the above test like this using FsCheck.Xunit:

 open FsCheck open FsCheck.Xunit [<Property>] let ``Multiplication is distributive`` () = Arb.generate<DoNotSize<int64>> |> Gen.map (fun (DoNotSize x) -> x) |> Gen.three |> Arb.fromGen |> Prop.forAll <| fun (x, y, z) -> let res1 = x * (y + z) let res2 = x * y + x * z res1 = res2 

This may hypothetically fail due to overflow, but after starting about 1,000,000 cases, I have not yet seen it fail.

The generator, however, does look like it is picking values ​​from the full range of 64-bit integers:

 > Arb.generate<DoNotSize<int64>> |> Gen.sample 1 10;; val it : DoNotSize<int64> list = [DoNotSize -28197L; DoNotSize -123346460471168L; DoNotSize -28719L; DoNotSize -125588489564554L; DoNotSize -29241L; DoNotSize 7736726437182770284L; DoNotSize -2382327248148602956L; DoNotSize -554678787L; DoNotSize -1317194353L; DoNotSize -29668L] 

Note that although I bind the size argument from Gen.sample to 1 , it selects “arbitrarily” large positive and negative values.