Remember IO?

Last time we looked at how IO works in Haskell

Functions that perform IO operations have

IO

in their return type:

putStr    :: String -> IO ()
getLine   :: IO String
writeFile :: FilePath -> String -> IO ()
readFile  :: FilePath -> IO String

We can build larger IO functions using the

do

-notation:

copyFile :: FilePath -> FilePath -> IO ()
copyFile from to = do text <- readFile from
                      writeFile to text

We can not hide a call to an IO function inside a pure function.
Pure functions have no side effects. Same argument ==> same result.

Random testing with QuickCheck

prop_example :: Integer -> Bool
prop_example n = (n+3)^2 == n^2 + 6*n + 9

quickCheck prop_example +++ OK, passed 100 tests.
QuickCheck generates 100 random integers. How is this done?

Test data generators

QuickCheck uses a type
```
Gen a
```
for random test data generators and provides an overloaded function with instances available for most predefined types:
```
arbitrary :: Arbitrary a => Gen a
```
There are also functions that let you have a look at the random values generators produce:

sample   :: Show a => Gen a -> IO ()
sample'  :: Gen a -> IO [a]
generate :: Gen a -> IO a

Sampling test data

sample' (arbitrary::Gen Integer) [0,2,-1,-6,2,4,4,-11,14,9,3] sample' (arbitrary::Gen Integer) [0,2,-2,-3,5,4,-9,-5,16,-13,-6]
Same argument, but different results.

Sampling Bool

sample' (arbitrary::Gen Bool) [True,False,True,False,False,True,True,False,False,True,True]
Two tests would be enough...

Sampling Lists

sample (arbitrary::Gen [Integer]) [] [2] [-3] [4] [7,-2,-3,-8,5,8,-5] [3,-4] [-10,0,7,1,2,5,-12,-3,8,-7,10,11] [-2,3,-5,-8,-3,-2,14,6,1,6,7,-2,-3,-12] [15,-13,15,-11,-11,-6,13,16,7,-13,-8,-2,-13,13,3,-13] [5,-15,5,13,12,1,-3,4,-13,-15,-7,18,-9,-18] [-8,-20,-10,-4,6,5,-15]
Short lists first, then longer lists.

Functions for creating generators

From the QuickCheck library:

elements  :: [a] -> Gen a
oneof     :: [Gen a] -> Gen a
frequency :: [(Int,Gen a)] -> Gen a
listOf    :: Gen a -> Gen [a]
vectorOf  :: Int -> Gen a -> Gen [a]
choose    :: Random a => (a,a) -> Gen a

```
choose
```
uses functions from the module System.Random.

Sampling generators

Testing
```
choose
```
and
```
listOf
```
sample' (choose ('a','z')) "ozlikagbygw" sample' (listOf (choose ('a','z'))) ["","t","xviy","","hfpkft","","iiisswjvrkg","suoz","slfosefhofpgla","","l"]

Generating a Suit

Remember the type Suit?

data Suit = Spades | Hearts | Diamonds | Clubs
          deriving (Eq,Show)

The function

elements

is perfect for generating random suits.

rSuit :: Gen Suit
rSuit = elements [Spades,Hearts,Diamonds,Clubs]

Generating a Rank

Remember the type `Rank`?

data Rank = Numeric Integer | Jack | Queen | King | Ace
            deriving (Eq,Show)

We could create a generator with
```
elements
```
and a list of all ranks. Here another way that gives us more flexibility:

rNumeric :: Gen Rank
rNumeric = elements [Numeric n|n<-[2..10]]

rFaceCard :: Gen Rank
rFaceCard = elements [Jack,Queen,King,Ace]

rRank_v1 :: Gen Rank
rRank_v1 = oneof [rNumeric,rFaceCard]

rRank :: Gen Rank
rRank = frequency [(9,rNumeric),(4,rFaceCard)]

```
rRank_v1
```
and
```
rRank
```
give different distributions...

Generating a Card?

Remember the type

Card

data Card = Card Rank Suit
            deriving (Eq,Show)

```
rCard :: Gen Card
rCard = (??)
```
Here we want to reuse
```
rSuit
```
and
```
rRank
```
. How can we do what?
Are there some functions to combine generators in the QuickCheck library?

Gen vs IO

The type
```
Gen a
```
is for functions that generate random values of type
```
a
```
.
The type
```
IO a
```
is for functions that performs IO operations and return values of type
```
a
```
.
But both are monads and the
```
do
```
-notation can be used with any monad!
- We can build larger generator functions from smaller ones in the same way as we build larger IO functions from smaller ones.

Generating a Card

rCard :: Gen Card
rCard = do r <- rRank
           s <- rSuit
           return (Card r s)

More examples

Other examples illustrating that
```
do
```
and
```
return
```
can be used with type
```
Gen a
```
, not just with type
```
IO a
```
.
sample' (return 5::Gen Integer) [5,5,5,5,5,5,5,5,5,5,5]
Remember
```
doTwice
```
?
sample' (doTwice (arbitrary::Gen Integer)) [(0,0),(2,-1),(1,1),(6,1),(-4,-1),(4,0), (-1,7),(4,11),(10,8),(18,-5),(11,14)]

More examples 2

evenInteger :: Gen Integer
evenInteger = do n <- arbitrary
                 return (2*n)

nonNegative :: Gen Integer
nonNegative = do n <- arbitrary
                 return (abs n)

sample' evenInteger [0,2,-4,0,-2,6,-10,-22,22,-2,18] sample' nonNegative [0,1,0,3,2,0,1,9,8,2,20]

Generating a Hand

Remember the type `Hand`?

data Hand = Empty | Add Card Hand
            deriving (Eq,Show)

rHand :: Gen Hand
rHand = oneof [return Empty,
               do c <- rCard
                  h <- rHand
                  return (Add c h)]

Note:
```
rHand
```
is recursive!
Note: does it generate sensible hands?

Telling QuickCheck which generator to use

The

Rank

type contains values that are not valid ranks.

validRank :: Rank -> Bool
validRank (Numeric n) = 2<=n && n<=10
validRank _ = True

We can use
```
validRank
```
to test that
```
rRank
```
is correct!

Telling QuickCheck which generator to use

Two ways to test if all generated cards are valid:

Using the function
```
forAll
```
:

prop_all_validRank_1 = forAll rRank validRank

Using
```
arbitrary
```
:

instance Arbitrary Rank where
  arbitrary = rRank

prop_all_validRank_2 r = validRank r

Arbitrary

class Arbitrary where
  arbitrary :: Gen a
  shrink :: a -> [a]

instance Arbitrary Bool
instance Arbitrary Int
instance Arbitrary Integer
-- many more instances...

It specificies a default test data generator for each type.
It lets us test functions without specifying generators explicitly.
The function
```
forAll
```
lets us use other generators.

Test Data Distribution

How do you know if a test has tested all the important cases?

prop_all_validRank_3 r = collect r (validRank r)

quickCheck prop_all_validRank_3 +++ OK, passed 100 tests: 12% Numeric 10 11% Numeric 7 10% Ace 9% Numeric 6 9% Numeric 4 9% King ... 1% Jack
Using
```
collect r
```
doesn't change the test, but collects the values of
```
r
```
that were tested.

Distribution of Hands

instance Arbitrary Hand where
  arbitrary = rHand

size Empty = 0
size (Add c h) = 1+size h

prop_Hand h = collect (size h) True

Distribution of Hands

quickCheck prop_Hand +++ OK, passed 100 tests: 47% 0 20% 1 13% 2 11% 3 4% 4 2% 5 1% 7 1% 6 1% 12
In general 50% empty hands and only 25% hands with more than one card.

More big Hands

We can increase the probability that a non-empty hand is generated.

rHand :: Gen Hand
rHand = frequency [(1,return Empty),
                   (4,do c <- rCard
                         h <- rHand
                         return (Add c h))]

More big Hands

quickCheck prop_Hand +++ OK, passed 100 tests: 19% 1 19% 0 13% 2 10% 4 8% 6 6% 3 5% 9 5% 5 3% 7 ...
In general 20% empty hands, 16% with one card, ...

Testing properties of algorithms

Testing insert

The function
```
insert
```
inserts a value at the right place in an ordered list
insert 'c' "abdef" "abcdef"
The output list is ordered if the input list is ordered.
How do we test it?

Testing insert, first try

prop_insert_1 :: Integer -> [Integer] -> Bool
prop_insert_1 x xs = isOrdered (insert x xs)

quickCheck prop_insert_1 *** Failed! Falsifiable (after 3 tests and 4 shrinks): 0 [0,-1]
If the input list isn't ordered, the output list won't be ordered. Bad test.

Testing insert, second try

prop_insert_2 :: Integer -> [Integer] -> Property
prop_insert_2 x xs = isOrdered xs ==> isOrdered (insert x xs)

quickCheck prop_insert_2 *** Gave up! Passed only 68 tests.
The probability that a random list is ordered is very low. (Can we test this?)
We need a test generator for ordered lists!

Testing ordered lists

Fortunately, QuickCheck provides

orderedList :: (Arbitrary a, Ord a) => Gen [a]

Two ways to use it

prop_insert_3 x =
    forAll orderedList (\xs->isOrdered (insert x xs))

prop_insert_4 x (Ordered xs) = isOrdered (insert x xs)

OrderedList

prop_insert_4 x (Ordered xs) = isOrdered (insert x xs)

This works, because QuickCheck also provides

data OrderedList a = Ordered [a]

instance (Ord a,Arbitrary a)
         => Arbitrary (OrderedList a) where
  arbitrary = Ordered <$> orderedList

Exercise: how would you define
```
orderedList
```
?
- See self-study exercises, Week 4, assignment 6.

Summary

We have seen how generate test data for QuickCheck.
- For our own data types (
```
Card
```
  , etc)
- How to deal with invariants (ordered lists)
- More info: QuickCheck documentation
We have also seen that
- ```
IO
```
  and
```
Gen
```
  are two examples of monads.
- The
```
do
```
  -notation can be used for both
```
IO
```
  and
```
Gen
```
  (and for other monads).
- Later we will see other examples of monads and how to define your own monads.

Test Data Generators

Remember IO?

Last time we looked at how IO works in Haskell

Random testing with QuickCheck

Test data generators

Sampling test data

Sampling Bool

Sampling Lists

Functions for creating generators

Sampling generators

Generating a Suit

Generating a Rank

Remember the type `Rank`?

Generating a Card?

Remember the type
Card
?

Gen vs IO

Generating a Card

More examples

More examples 2

Generating a Hand

Remember the type `Hand`?

Telling QuickCheck which generator to use

Telling QuickCheck which generator to use

Two ways to test if all generated cards are valid:

Arbitrary

Test Data Distribution

Distribution of Hands

Distribution of Hands

More big Hands

We can increase the probability that a non-empty hand is generated.

More big Hands

Testing properties of algorithms

Testing insert

Testing insert, first try

Testing insert, second try

Testing ordered lists

OrderedList

Summary

Test Data Generators

Remember IO?

Last time we looked at how IO works in Haskell

Random testing with QuickCheck

Test data generators

Sampling test data

Sampling Bool

Sampling Lists

Functions for creating generators

Sampling generators

Generating a Suit

Generating a Rank

Remember the type Rank?

Generating a Card?

Remember the type Card?

Gen vs IO

Generating a Card

More examples

More examples 2

Generating a Hand

Remember the type Hand?

Telling QuickCheck which generator to use

Telling QuickCheck which generator to use

Two ways to test if all generated cards are valid:

Arbitrary

Test Data Distribution

Distribution of Hands

Distribution of Hands

More big Hands

We can increase the probability that a non-empty hand is generated.

More big Hands

Testing properties of algorithms

Testing insert

Testing insert, first try

Testing insert, second try

Testing ordered lists

OrderedList

Summary

Remember the type `Rank`?

Remember the type
Card
?

Remember the type `Hand`?