Testing, Debugging, and Verification TDA567/DIT082, LP2, HT2017
Lab 1 - Software Testing

This lab is divided into three parts:

  • Manual Black-box Testing
  • Manual White-box Testing
  • Debugging

The programming language of the assignments is Java. When deriving test cases you are supposed to use JUnit to organise and run your test cases.

Installation instruction for JUnit: here.
How to use JUnit in Eclipse: here.
And how to use JUnit in IntelliJ: here.

1. Black-box Testing

The assignment is about a class called WorkSchedule. The purpose of the class is to manage the schedule for the employees of a company. The time is divided in units of 1 hour and the hours are simply identified by integers (Note that this is an unrealistic simplification). For each hour, the schedule stores the number employees which is needed at that time. It could for instance be that during week-days the need is that 5 employees work at the same time, but less during nights and weekend. For each hour the schedule also stores the names of the employees who have been assigned to work at that hour.

The constructor and methods of the class are: 

  public WorkSchedule(int size) { ... }

creates a schedule which contains the hours 0,1,2,...,size - 1 where for each hour the number of needed employees is set to zero and there are no employees assigned to it

  public Hour readSchedule(int time) { ... }

gives back an object of the class Hour, which has two fields:
requiredNumber of type int is the required number of employees working at hour time.
workingEmployees of type String[] is the names of the employees who have so far been assigned to work at hour time. 

  public void setRequiredNumber(int nemployee, int starttime, int endtime) { ... }

sets the number of required working employees to nemployee for all hours in the interval starttime to endtime. 

  public boolean addWorkingPeriod(String employee, int starttime, int endtime) { ... }

schedules employee to work during the hours from starttime to endtime. 

  public String[] workingEmployees(int starttime, int endtime) { ... }

returns a list of all employees working at some point during the interval starttime to endtime. 

  public int nextIncomplete(int currenttime) { ... }

returns the closest time starting from currenttime for which the required amount of employees has not yet been scheduled.

The constructor (WorkSchedule) and the methods readSchedule, setRequiredNumber, nextIncomplete, are trusted to be correct and do not need to be tested. Note that readSchedule can be used to examine the state of a class instance.

A compiled implementation of the class can be found in the following Java archive:

> WorkSchedule.jar

Your task: For each specification provided below (i.e. specification for the methods addWorkingPeriod, and workingEmployees):

  1. Specify (the domain and) the input space for each argument to the method (including the object of that method).
  2. Divide this input space into partitions, based on the specification.
  3. Implement at least one test case for each partition, and possibly additional border cases.,
  4. For each test case, give a comment which describes what partition it tests.
a) addWorkingPeriod
requires:
  employee is a non-null string
ensures:
  if starttime < 0 or endtime >= size or starttime > endtime then
    returns false and the schedule is unchanged
  otherwise
    if for any hour in the interval starttime to endtime the length of workingEmployees is equal to requiredNumber then
      returns false and the schedule is unchanged
    otherwise
      if for any hour in the interval starttime to endtime there is a string in workingEmployees which equal employee then
        returns false and the schedule is unchanged
      otherwise
        returns true,
        for i between starttime and endtime, workingEmployees contain a string equal to employee and
        the rest of the schedule is unchanged
b) workingEmployees
requires:
  starttime >= 0 and endtime < size
ensures:
  if starttime <= endtime then
    returns an array with distinct strings -- a string appears in the return array if and only if
    it appears in the workingEmployees of at least one hour in the interval starttime to endtime
  otherwise
    returns an empty array
  and in either case the schedule is unchanged

Reporting test cases

In a report file, you must explain and motivate why you divided the test cases in the way you did. You must show how you defined the input space, and how you partitioned it.

Example: Given the method: 

  public String m(int x);
With the specification: 
  requires:  x != 0
  ensures:   if x > 0 then
               the method returns the String "High"
             otherwise
               the method returns the String "Low"
Your report should read something like: 
  Input space: x != 0 (or: "x" in { MIN_INT, ..., -1, 1, ..., MAX_INT } )

  Partition #1: x > 0 (or: "x" in { 1, ..., MAX_INT } )
    test_m_part1: input: "x = 155", expected: "m(x) = "High""
  Partition #2: x < 0 (or: "x" in { MIN_INT, ..., -1 } )
    test_m_part2: input: "x = -18", expected: "m(x) = "Low""

  Border cases:
    test_m_border1: input: "x = MIN_INT", expected: "m(x) = "Low""
    test_m_border2: input: "x = -1",      expected: "m(x) = "Low""
    test_m_border3: input: "x = 1",       expected: "m(x) = "High""
    test_m_border4: input: "x = MAX_INT", expected: "m(x) = "High""

  Found bugs: None.


2. White-box Testing

For the assignments in part 2 you will be using the source code of the methods. For each method you should first derive a suite of test cases which together have full statement coverage of the source code. Then you should write additional test cases (if any are needed) which together with the ones already there have full branch coverage of the code. The test cases should be executable with JUnit just like in part 1 of the lab. If any of the test cases for a method fails, try to find the bug and correct the source code.

Keep in mind that you may need a method, A, to set up the test of another method, B. If the method A has a bug it can be very tricky to test method B. Wherever possible you can first fix the bug in a method and then do the tests that use it in the setup (or test oracle). Also keep in mind that a bug in the implementation may imply that full statement or branch coverage is not possible. This it in itself an indication of the bug's existence. In such case, it's enough if you have full coverage after fixing the bug.

The assignment is about the class Set.

> Set

Set represents sets of integers. The elements of a set are stored in an ArrayList. They are sorted and without duplicates to speed up some operations. Two methods might need an explanation: 

  public void section(Set s) { ... }

removes from this set any element that is equal to an element in s. 

  public boolean containsArithTriple() { ... }

returns true iff there are three elements, x, y and z, in this set such that y - x = z - y.

Note that you can use toArray to easily examine the state of a class instance.

a) Use statement and branch coverage to derive test cases for insert.

b) Use statement and branch coverage to derive test cases for member.

c) Use statement and branch coverage to derive test cases for section.

d) Use statement and branch coverage to derive test cases for containsArithTriple.

e) If you find any bugs, try to correct them in the implementation. Also keep in mind that it might be your specification that is incorrect. Make a new version of Set.java which passes all your tests.

3. Debugging

When an array is sorted, finding an element in that array becomes possible in logarithmic time by performing a binary search. The class BinarySearch implements a binary search for arrays of integers. However, there is a bug in the implementation of the method search. In the method main, replace the value 4 in the variable value by any other of the elements in array, and search will not be able to find it.

Your task is to debug the previous implementation in order to find the error in method search. In your report for this exercise you must include a description of how you have debugged the previous method (e.g. break points used, etc).

Reporting

Upload an archive LAB1.zip (or rar, tar.gz, tar as you wish) using the Fire report system. The archive must have the following structure:

LAB1.zip
 |
 |-ex1
 | |- <your_1st_test_class>.java
 | |- ...
 | |- <your_nth_test_class>.java
 | |- WorkScheduleTestSuite.java
 | \- report.txt
 |
 |-ex2
 | |- <your_1st_Set_test_class>.java
 | |- ...
 | |- <your_mth_Set_test_class>.java
 | |- SetTestSuite.java
 | |- Set.java
 | \- report.txt
 |
 \-ex3
   |- BinarySearch.java
   \- report.txt
  • ex1, ex2, ex3 are directories
  • ex1/WorkScheduleTestSuite.java is the test suite class for Exercise 1. It looks like the following replacing the contents within angle brackets < > with your own class names in the same directory: 
    	import org.junit.runner.RunWith;
	import org.junit.runners.Suite;
	@RunWith(Suite.class)
	@Suite.SuiteClasses({
		<your_1st_test_class>.class,
		...
		<your_nth_test_class>.class
	})
	public class WorkScheduleTestSuite {
	// nothing goes here	 
	}
    Executing with JUnit the WorkScheduleTestSuite class makes JUnit run the test cases contained in the classes specified inside the @Suite.SuiteClasses annotation arguments.
  • ex2/SetTestSuite.java is the test suite class for Exercise 2. It looks like the following replacing the contents within angle brackets < > with your own class names in the same directory: 
    	import org.junit.runner.RunWith;
	import org.junit.runners.Suite;
	@RunWith(Suite.class)
	@Suite.SuiteClasses({
		<your_1st_Set_test_class>.class,
		...
		<your_mth_Set_test_class>.class
	})
	public class SetTestSuite {
	// nothing goes here	 
	}
    Executing with JUnit the SetTestSuite class makes JUnit run the test cases contained in the classes specified inside the @Suite.SuiteClasses annotation arguments.
  • ex1/<your_1st to nth_test_class>.java are all classes implementing test cases for exercise 1 (replacing the contents within angle brackets < > with your own class names)
  • ex2/<your_1st to mth_Set_test_class>.java are all classes implementing test cases for exercise 2 (replacing the contents within angle brackets < > with your own class names)
  • report.txt is the report file (where you have to motivate how you engineered the test and report bugs you found / fixed).
Remark: No need to be more complex than this; avoid package declarations, avoid to add unnecessary files such as hidden files or folders produced by your text editor, for instance.



Home | Course | Schedule | Exam | Exercises | Labs | Evaluation | Tools Srinivas Pinisetty, Nov 7, 2017