"QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs" Koen Claessen and John Hughes (ICFP'2000).

Web site:

http://www.cse.chalmers.se/~rjmh/QuickCheck/

Introductory tutorial:

https://www.schoolofhaskell.com/user/pbv/an-introduction-to-quickcheck-testing

- a library for randomly-testing of Haskell programs
- user writes general
*properties*rather than specific tests - QuickCheck tests them with randomly-generated data

```
import Test.QuickCheck
prop_rev_rev xs = reverse (reverse xs) == xs
where types = xs::[Int]
```

Ask GHCi to check this with randomly generated lists:

```
Main> quickCheck prop_rev_rev
OK, passed 100 tests.
```

If the property fails, then QuickCheck prints out a *counter-example*.

```
import Test.QuickCheck
prop_rev_id xs = reverse xs == xs -- WRONG!
where types = xs::[Int]
Main> quickCheck prop_rev_id
*** Failed! Falsifiable
(after 6 tests and 6 shrinks):
[0,1]
```

- properties can be ordinary Haskell boolean functions
- convention: names start with
`prop_`

- parameters are implicitly
*universally quantified* - polymorphic functions should be tested
*with a specific type*

For the previous example:

$$ \text{reverse}~(\text{reverse}~ xs) = xs, \quad
\forall xs :\hspace{-0.1ex}: [\alpha] $$

We choose to test for `xs :: [Int]`

```
insert :: Ord a => a -> [a] -> [a]
insert x [] = [x]
insert x (y:ys)
| x<y = x:y:ys
| otherwise = y:insert x ys
```

Let us test that *insertion preserves list ordering*, i.e.

*a**s**c**e**n**d**i**n**g* *x**s* ⟹ *a**s**c**e**n**d**i**n**g* (*i**n**s**e**r**t* *x* *x**s*), ∀*x*, *x**s*

The operator `==>`

allows expressing such *conditional properties*.

```
prop_ins_ord x xs
= ascending xs ==> ascending (insert x xs)
where types = x::Int
-- auxiliary definition;
-- check values are in ascending order
ascending :: Ord a => [a] -> Bool
ascending ...
```

```
> quickCheck prop_ins_ord
*Main Data.List> quickCheck prop_ins_ord
*** Gave up! Passed only 72 tests.
```

What happened?

- the operator
`==>`

discards test cases that do not satisfy the pre-condition - fewer than 100 test cases passed this test
- QuickCheck gave up trying to generate more cases
- maybe testing wasn't sufficiently thorough...?

We can use `verboseCheck`

to show the actual test cases:

```
Main> verboseCheck prop_ins_ord
Passed:
1
[]
Skipped (precondition false):
-1
[1,-2]
....
```

This *too* verbose; it is better to *aggregate statistical* data on test cases.

```
prop_ins_ord x xs
= collect (length xs) $
ascending xs ==> ascending (insert x xs)
where types = x::Int
Main> quickCheck prop_ins_ord
*** Gave up! Passed only 75 tests:
38% 0
29% 2
25% 1
4% 4
2% 3
```

38% were the *empty list*

25% were *single element lists*

29% were *two element lists*

Only 6% were 3 or 4 elements!

Why?

Because most randomly-list generated lists of larger sizes are *not* ordered --- the pre-condition biases test cases to "small" lists.

We can restrict test cases using explicit *generators*:

`forAll <generator> $ \pattern -> <property>`

For our example we can use the `orderedList`

generator for lists of values in ascending order (pre-defined in the library):

```
prop_ins_ord2 x
= forAll orderedList $
\xs -> ascending (insert x xs)
where types = x :: Int
```

A custom generator ensures better distributed test cases.

```
prop_ins_ord2 x =
forAll orderedList $ \xs ->
collect (length xs) $ ascending (insert x xs)
where types = x :: Int
Main> quickCheck prop_ins_ord2
6% 0
5% 1
4% 32
4% 22
4% 2
...
```

- QuickCheck exports some
*basic generators*and some*combinators*for making new ones `Gen a`

is the type for generators of`a`

s`Gen`

is a*monad*combining:- a
*pseudo-random generation seed*; - a
*size measure*for data structures (i.e. list length).

- a
- Hence: we can use
*do-*or*applicative notation*for defining generators

```
choose :: Random a => (a,a) -> Gen a
-- random choice from interval
elements :: [a] -> Gen a
-- random choice from list
```

Examples:

```
dice :: Gen Int
dice = choose (1,6)
vowel :: Gen Char
vowel = elements "aeiouAEIOU"
```

```
-- using do-notation
sum_dice :: Gen Int
sum_dice = do d1 <- dice
d2 <- dice
return (d1+d2)
-- using applicative operators
sum_dice' :: Gen Int
sum_dice' = (+) <$> dice <*> dice
```

We can use `sample`

to show random samples of a generator.

```
Main> sample sum_dice
3
7
8
4
8
7
5
6
2
3
6
```

```
oneof :: [Gen a] -> Gen a
-- uniform random choice
frequency :: [(Int,Gen a)] -> Gen a
-- random choice with frequency
```

Examples:

```
fruit = elements ["apple", "orange", "banana"]
sweet = elements ["cake", "ice-cream"]
-- fruit and sweets equally probable
desert = oneof [fruit, sweet]
-- healthy option: 4/5 fruits, 1/5 sweets
healthy = frequency [(4,fruit),(1,sweet)]
```

```
Main> sample healthy
"orange"
"ice-cream"
"apple"
"banana"
"ice-cream"
"apple"
"apple"
"banana"
"orange"
"banana"
"orange"
```

We can write a *custom generator* using basic combinators and do-notation.

```
data Person = Person String Int
genPerson :: Gen Person
genPerson =
do name <- elements ["Alice", "Bob",
"David", "Carol"]
age <- choose (10, 99)
return (Person name age)
```

```
Main> sample genPerson
Person "Alice" 77
Person "David" 87
Person "Alice" 67
Person "Bob" 51
Person "David" 91
Person "David" 36
Person "Alice" 47
Person "Alice" 84
Person "Carol" 36
Person "Bob" 94
```

A *default generator* for a type is specified in the `Arbitrary`

type class.

```
class Arbitrary a where
arbitrary :: Gen a
```

The library defines instances for all basic and parametric types:

```
instance Arbitrary Int
instance Arbitrary Bool
...
instance Arbitrary a => Arbitrary [a]
instance (Arbitrary a,
Arbitrary b) => Arbitrary (a,b)
...
```

We can define instances of `Arbitrary`

for new types:

```
instance Arbitrary Person where
arbitrary = genPerson
genPerson :: Gen Person
genPerson = ...
```

```
listOf :: Gen a -> Gen [a]
-- lists of random length
listOf1 :: Gen a -> Gen [a]
-- non-empty lists of random length
vectorOf :: Int -> Gen a -> Gen [a]
-- list of the given length
```

```
Main> sample (vectorOf 5 $ elements "aeiou")
"ueuuo"
"aioua"
"iouui"
"iaoue"
"aeaii"
"iiuio"
"uuaoe"
"ieeue"
"ouiei"
"uoeoo"
"ueaeo"
```

- QuickCheck begins by generating "small" test data, gradually increasing a
*size parameter*. - the meaning of the size parameter is data-dependent:
- ignored for fixed-size types (e.g.
`Bool`

or`Char`

); - is the
*length*for lists - is the
*magnitude*for`Int`

and`Integer`

- ...

- ignored for fixed-size types (e.g.
- we can use
`sized`

to access the size parameter:

` sized :: (Int -> Gen a) -> Gen a`

- useful for writing generators of
*recursive data structures*

```
data Tree a = Branch a (Tree a) (Tree a)
| Leaf
```

```
instance Arbitrary a =>
Arbitrary (Tree a) where
arbitrary = sized genTree
genTree :: Arbitrary a => Int -> Gen (Tree a)
genTree size
-- size ~ number of branch nodes
| size>0 = do v <- arbitrary
l <- genTree (size`div`2)
r <- genTree (size`div`2)
return (Branch v l r)
| otherwise = return Leaf
```

```
genTree' :: Arbitrary a => Int -> Gen (Tree a)
genTree' size
-- `size' ~ upper bound of inner nodes
| size>0 = frequency [(4, genBranch),
(1, return Leaf)]
| otherwise = return Leaf
where
genBranch = do v <- arbitrary
l <- genTree' (size`div`2)
r <- genTree' (size`div`2)
return (Branch v l r)
```

- a property
*fails*when QuickCheck finds a*first*counter-example - however, randomly-generated data typically contains a lot of "noise"
- therefore it is a good idea to
*simplify counter-examples*before reporting them - this is called
*shrinking*in QuickCheck

```
prop_wrong xs ys = reverse (xs ++ ys) ==
reverse xs ++ reverse ys
where types = (xs::[Int], ys::[Int])
```

QuickCheck randomly generates one counter-example, e.g.: `xs = [2,2]`

and `ys = [1]`

.

Shrinking tries to simplify the lists recursively:

- taking sub-lists of
`xs`

and`ys`

; - shrinking the values inside the list.

This process is iterated until no "smaller" counter-example is found.

`xs=[0]`

, `ys=[1]`

obtained after two shrinking steps; further shrinking does not yield counter-examples (**local minimum**).

`shrink`

functionThe shrinking function is defined in the `Arbitrary`

type class.

```
class Arbitrary a where
arbitrary :: Gen a -- generator
shrink :: a -> [a] -- shrink function
shrink _ = [] -- default: no shrinking
```

`shrink`

should give a (finite) list of "smaller" values to test for counter-examples.

The default implementation yields an empty list --- i.e. shrinking is disabled.

```
data Tree a = Branch a (Tree a) (Tree a)
| Leaf
```

Heuristic to shrink a branching node:

- replace it by a leaf;
- replace it by the left or right sub-trees;
- shrink the left or right sub-trees;
- shrink the value.

No further shrinking when we reach a leaf.

```
instance Arbitrary a =>
Arbitrary (Tree a) where
arbitrary = ...
shrink = shrinkTree
shrinkTree :: Arbitrary a => Tree a -> [Tree a]
shrinkTree (Branch v l r) =
[Leaf, l, r] ++
[Branch v l' r | l'<-shrinkTree l] ++
[Branch v l r' | r'<-shrinkTree r] ++
[Branch v' l r | v'<-shrink v]
shrinkTree _ = [] -- no shrinking for leaves
```

- The
`shrink`

function should yield "single step" smaller values - QuickCheck will iterate your
`shrink`

function until it stabilizes - Let
*u*≻*v*iff*v*∈ shrink*u*; for shrinking to terminate there should be*no infinite descending chains**v*_{0}≻*v*_{1}≻ ⋯ ≻*v*_{n}≻*v*_{n + 1}≻ ⋯

- Tree rotations (used in re-balancing algorithms)
- Property: rotations should preserve values in the tree

```
rotateL, rotateR :: Tree a -> Tree a
rotateL (Branch x t1 (Branch y t2 t3))
= Branch y (Branch x t1 t2) t3
rotateL t = t -- nothing to do
rotateR (Branch x (Branch y t1 t2) t3)
= Branch x t1 (Branch y t2 t3)
rotateR t = t -- nothing to do
```

```
prop_rotateL_inv t
= toList t == toList (rotateL t)
where types = t :: Tree Int
prop_rotateR_inv t
= toList t == toList (rotateR t)
where types = t :: Tree Int
-- list tree values in order
toList :: Tree a -> [a]
toList Leaf = []
toList (Branch v l r)
= toList l ++ [v] ++ toList r
```

```
*Main> quickCheck prop_rotateL_inv
+++ OK, passed 100 tests.
Main> quickCheck prop_rotateR_inv
*** Failed! Falsifiable
(after 3 tests and 3 shrinks):
Branch 1 (Branch 0 Leaf Leaf) Leaf
```

We found an error!

`rotateR`

is incorrect (`x`

and`y`

are switched around)- because of shrinking, the counter-example is
*minimal*

Using `newtype`

allows customizing the generators for specific uses.

`data Expr = Var String | ...`

vs.

```
data Expr = Var Name | ...
newtype Name = Name String
instance Arbitrary Name where
arbitrary = do n<-elements ["a","b","c","d"]
return (Name n)
```

```
newtype NonNegative a = NonNegative a
newtype NonZero a = NonZero a
newtype Positive a = Positive a
newtype NonEmptyList a = NonEmpty [a]
newtype OrderedList a = Ordered [a]
```

Allow restricting the range of generated values:

```
prop_add (Positive x) y = x+y > y
where types = (x::Integer,y::Integer)
prop_len (NonEmpty xs) = length xs > 0
where types = xs :: [Int]
```