“QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs” Koen Claessen and John Hughes (ICFP’2000).

Web site:

http://www.cse.chalmers.se/~rjmh/QuickCheck/

Introductory tutorial:

https://www.schoolofhaskell.com/user/pbv/an-introduction-to-quickcheck-testing

- a library for randomly-testing of Haskell programs
- user writes general
*properties*rather than specific tests - QuickCheck tests them with randomly-generated test cases

```
import Test.QuickCheck
prop_rev_rev xs = reverse (reverse xs) == xs
where types = xs::[Int]
```

Ask GHCi to check this with randomly generated lists:

```
Main> quickCheck prop_rev_rev
OK, passed 100 tests.
```

If the property fails, then QuickCheck prints out a *counter-example*.

```
import Test.QuickCheck
prop_rev_id xs = reverse xs == xs -- WRONG!
where types = xs::[Int]
Main> quickCheck prop_rev_id
*** Failed! Falsifiable
(after 6 tests and 6 shrinks):
[0,1]
```

- properties are expressed as ordinary Haskell boolean functions
- convention: property names start with
`prop_`

- parameters are
*universally quantified* - polymorphic functions should be tested
*with a specific type*

For the previous example:

$$ \text{reverse}~(\text{reverse}~ xs) = xs, \quad
\forall xs :\hspace{-0.1ex}: [\alpha] $$

We choose to test for `xs :: [Int]`

```
insert :: Ord a => a -> [a] -> [a]
insert x [] = [x]
insert x (y:ys)
| x<y = x:y:ys
| otherwise = y:insert x ys
```

Let us test that *insertion preserves the list ordering*:

$$
\text{ascending}~xs \implies \text{ascending}~(\text{insert}~x~xs),
\quad
\forall x, xs $$

The operator `==>`

allows writing such *conditional properties*.

```
prop_ins_ord x xs
= ascending xs ==> ascending (insert x xs)
where types = x::Int
-- auxiliary definition;
-- check elements are in ascending order
ascending :: Ord a => [a] -> Bool
ascending (x:x':xs) = x<=x' && ascending (x':xs)
ascending _ = True -- empty and singleton lists
```

```
> quickCheck prop_ins_ord
*Main Data.List> quickCheck prop_ins_ord
*** Gave up! Passed only 72 tests.
```

What happened?

`==>`

*discards test cases*that do not satisfy the pre-condition- fewer than 100 test cases passed this test
- QuickCheck gave up trying to generate more cases
- maybe testing wasn’t sufficiently thorough…?

We can use `verboseCheck`

to show the actual test cases:

```
Main> verboseCheck prop_ins_ord
Passed:
1
[]
Skipped (precondition false):
-1
[1,-2]
....
```

This *too* verbose; it is better to *aggregate statistical* data on test cases.

```
prop_ins_ord x xs
= collect (length xs) $
ascending xs ==> ascending (insert x xs)
where types = x::Int
Main> quickCheck prop_ins_ord
*** Gave up! Passed only 75 tests:
38% 0
29% 2
25% 1
4% 4
2% 3
```

38% were the *empty list*

25% were *single element lists*

29% were *two element lists*

Only 6% were 3 or 4 elements!

Why?

Because most randomly-list generated lists of larger sizes are *not* ordered — the pre-condition biases test cases to “small” lists.

We can restrict test cases using explicit *generators*:

`forAll <generator> $ \pattern -> <property>`

For our example we can use the `orderedList`

generator for lists of values in ascending order (pre-defined in the library):

```
prop_ins_ord2 x
= forAll orderedList $
\xs -> ascending (insert x xs)
where types = x :: Int
```

A custom generator ensures better distributed test cases.

```
prop_ins_ord2 x =
forAll orderedList $ \xs ->
collect (length xs) $ ascending (insert x xs)
where types = x :: Int
Main> quickCheck prop_ins_ord2
6% 0
5% 1
4% 32
4% 22
4% 2
...
```

- QuickCheck exports some
*basic generators*and some*combinators*for making new ones `Gen a`

is the type for generators of`a`

s`Gen`

is a*state monad*combining:- a
*pseudo-random generation seed*; - a
*size measure*for data structures (i.e. list length).

- a
- We can also use
*do-notation*when defining generators

```
choose :: Random a => (a,a) -> Gen a
-- random choice from interval
elements :: [a] -> Gen a
-- random choice from list
```

Examples:

```
sum_dice :: Gen Int
sum_dice = do d1 <- choose (1,6)
d2 <- choose (1,6)
return (d1+d2)
one_vowel :: Gen Char
one_vowel = elements "aeiouAEIOU"
```

We can use `sample`

to show random samples of a generator.

```
Main> sample sum_dice
3
7
8
4
8
7
5
6
2
3
6
```

```
oneof :: [Gen a] -> Gen a
-- uniform random choice
frequency :: [(Int,Gen a)] -> Gen a
-- random choice with frequency
```

Examples:

```
fruit = elements ["apple", "orange", "banana"]
sweet = elements ["cake", "ice-cream"]
-- fruit and sweets equally probable
desert = oneof [fruit, sweet]
-- healthy option: 4/5 fruits, 1/5 sweets
healthy = frequency [(4,fruit),(1,sweet)]
```

```
Main> sample healthy
"orange"
"ice-cream"
"apple"
"banana"
"ice-cream"
"apple"
"apple"
"banana"
"orange"
"banana"
"orange"
```

We can write a *custom generator* using basic combinators and do-notation.

```
data Person = Person String Int -- name & age
genPerson :: Gen Person
genPerson =
do name <- elements ["Alice", "Bob",
"David", "Carol"]
age <- choose (10, 99)
return (Person name age)
```

```
Main> sample genPerson
Person "Alice" 77
Person "David" 87
Person "Alice" 67
Person "Bob" 51
Person "David" 91
Person "David" 36
Person "Alice" 47
Person "Alice" 84
Person "Carol" 36
Person "Bob" 94
```

A *default generator* for a type is specified in the `Arbitrary`

type class.

```
class Arbitrary a where
arbitrary :: Gen a
```

The library defines instances for all basic and parametric types:

```
instance Arbitrary Int
instance Arbitrary Bool
...
instance Arbitrary a => Arbitrary [a]
instance (Arbitrary a,
Arbitrary b) => Arbitrary (a,b)
...
```

We can define an instance of `Arbitrary`

for our new types.

```
data Person = Person String Int
genPerson :: Gen Person
genPerson =
do name <- elements ["Alice", "Bob",
"David", "Carol"]
age <- choose (10, 99)
return (Person name age)
instance Arbitrary Person where
arbitrary = genPerson
```

```
listOf :: Gen a -> Gen [a]
-- lists of random length
listOf1 :: Gen a -> Gen [a]
-- non-empty lists of random length
vectorOf :: Int -> Gen a -> Gen [a]
-- list of the given length
```

```
Main> sample (vectorOf 5 $ elements "aeiou")
"ueuuo"
"aioua"
"iouui"
"iaoue"
"aeaii"
"iiuio"
"uuaoe"
"ieeue"
"ouiei"
"uoeoo"
"ueaeo"
```

- QuickCheck begins by generating “small” test cases, gradually increasing the size.
- the meaning of “size” is data-dependent:
- ignored for “fixed size” types (e.g.
`Bool`

or`Char`

); - is the
*length*for lists - is the
*magnitude*for`Int`

and`Integer`

- …

- ignored for “fixed size” types (e.g.
- we can use
`sized`

to access the size measure in our own generators - useful for generating
*recursive data structures*

` sized :: (Int -> Gen a) -> Gen a`

```
data Tree a = Branch a (Tree a) (Tree a)
| Leaf
```

```
instance Arbitrary a => Arbitrary (Tree a) where
arbitrary = sized genTree
-- `size' : approximate # inner nodes
genTree size
| size>0 = do v <- arbitrary
l <- genTree (size`div`2)
r <- genTree (size`div`2)
return (Branch v l r)
| otherwise = return Leaf
```

```
instance Arbitrary a => Arbitrary (Tree a) where
arbitrary = sized genTree'
-- `size' : upper bound on # inner nodes
genTree' size
| size>0 = frequency [(4, genBranch),
(1, return Leaf)]
| otherwise = return Leaf
where
genBranch = do v <- arbitrary
l <- genTree' (size`div`2)
r <- genTree' (size`div`2)
return (Branch v l r)
```

- a property
*fails*when QuickCheck finds a*first*counter-example - however, randomly-generated data typically contains a lot of “noise”
- therefore it is a good idea to
*simplify counter-examples*before reporting them - this is called
*shrinking*in QuickCheck

```
prop_wrong xs ys = reverse (xs ++ ys) ==
reverse xs ++ reverse ys
where types = (xs::[Int], ys::[Int])
```

QuickCheck randomly generates one counter-example, e.g.: `xs = [2,2]`

and `ys = [1]`

.

Shrinking tries to simplify the lists recursively:

- taking sub-lists of
`xs`

and`ys`

; - shrinking the values inside the list.

This process is iterated until no “smaller” counter-example is found.

We reached a “local minimum” `xs=[0]`

, `ys=[1]`

after two shrinking steps; this minimal counter-example is reported.

`shrink`

functionThe shrinking function is defined in the `Arbitrary`

type class.

```
class Arbitrary a where
arbitrary :: Gen a -- generator
shrink :: a -> [a] -- shrink function
shrink _ = [] -- default: no shrinking
```

The `shrink`

function should yield a (finite) list of “smaller” values to test for counter-examples.

The default implementation yields an empty list — i.e. no shrinking candidates.

```
data Tree a = Branch a (Tree a) (Tree a)
| Leaf
```

Heuristic to shrink a branching node:

- replace it by a leaf;
- replace it by the left or right sub-trees;
- shrink the left or right sub-trees;
- shrink the value.

No more shrinking when we reach a leaf.

```
instance Arbitrary a => Arbitrary (Tree a) where
arbitrary = ...
shrink = shrinkTree
shrinkTree :: Arbitrary a => Tree a -> [Tree a]
shrinkTree (Branch v l r) =
[Leaf, l, r] ++
[Branch v l' r | l'<-shrinkTree l] ++
[Branch v l r' | r'<-shrinkTree r] ++
[Branch v' l r | v'<-shrink v]
shrinkTree _ = [] -- no shrinking for leaves
```

- The
`shrink`

function should yield “single step” smaller values - QuickCheck will iterate your
`shrink`

function until it stabilizes - Let
*u*≻*v*iff*v*∈ shrink*u*; for shrinking to terminate there should be*no infinite descending chains**v*_{0}≻*v*_{1}≻ ⋯ ≻*v*_{n}≻*v*_{n + 1}≻ ⋯

- Tree rotations (used in re-balancing algorithms)
- Property: rotations should preserve values in the tree

```
rotateL, rotateR :: Tree a -> Tree a
rotateL (Branch x t1 (Branch y t2 t3))
= Branch y (Branch x t1 t2) t3
rotateL t = t -- nothing to do
rotateR (Branch x (Branch y t1 t2) t3)
= Branch x t1 (Branch y t2 t3)
rotateR t = t -- nothing to do
```

```
prop_rotateL_inv t
= toList t == toList (rotateL t)
where types = t :: Tree Int
prop_rotateR_inv t
= toList t == toList (rotateR t)
where types = t :: Tree Int
-- list tree values in order
toList :: Tree a -> [a]
toList Leaf = []
toList (Branch v l r)= toList l ++ [v] ++
toList r
```

```
*Main> quickCheck prop_rotateL_inv
+++ OK, passed 100 tests.
Main> quickCheck prop_rotateR_inv
*** Failed! Falsifiable
(after 3 tests and 3 shrinks):
Branch 1 (Branch 0 Leaf Leaf) Leaf
```

We found an error!

`rotateR`

is incorrect (`x`

and`y`

are switched around)- due to shrinking, the counter-example found is
*minimal*

Using `newtype`

allows customizing the generators for specific uses.

`data Expr = Var String | ...`

vs.

```
data Expr = Var Name | ...
newtype Name = Name String
instance Arbitrary Name where
arbitrary = do n<-elements ["a","b","c","d"]
return (Name n)
```

```
newtype NonNegative a
newtype NonZero a
newtype Positive a
newtype NonEmptyList a
newtype OrderedList a
```

Allow restricting the range of generated values:

```
prop_add (Positive x) y = x+y > y
where types = (x::Integer,y::Integer)
prop_len (NonEmptyList xs) = length xs > 0
where types = xs :: [Int]
```