The goals of this homework are to:
- familiarize you with code coverage concepts, tools, and metrics;
- practice writing thorough test suites to improve the quality of your code
Please keep track of how much time (in hours) it takes you to solve each exercise. Please download and complete the timeSpent.txt template (replace xxx with the number of hours you spent) and include this file with your deliverables.
You must use the provided answers.txt template to answer the questions.
The total number of points for this homework is 100.
Please be mindful of the Collaboration Policy. If you have any questions regarding the assignment requirements, ask on the discussion group. If you have any doubts about whether it's appropriate to ask something, check with the course staff. You are allowed to use Web search to find answers to questions.
The deadline for this homework is December 3, 2010, at 23:59 CET. Assignments (or parts thereof) handed in late will not be considered.
Part 1: Test Suite for Homework #3 (90 points)
In this part of the homework, you must write a test suite for the TA-developed solution of homework 3.
You can either base your test suite on the
revision 3 (November 17th), or on the
revision 4 (November 26th) of the code base. The revision 4 includes the fixes for the bugs you have found in between. Please specify in answers.txt which revision you use.
To get all points for this part 1, you must achieve the following coverage levels (on either of the revisions):
- 100% function coverage for each source file in the TA-developed solution
- 98% statement coverage overall, across all source files in the TA-developed solution
- 94% branch coverage overall, across all source files in the TA-developed solution
and, for the parts of the code that are not covered, you must identify them and explain why they are not covered. An explanation could look like this:
"The condition on the line 123 in the file MyClass.java cannot be false because it checks a class invariant xyz. As a result, line 124 cannot be exercised in a bug-free code."
You must stick to the following rules:
- Use CodeCover on the BC machines to measure the coverage (see this tutorial)
- Use JUnit4
- Place your test suite in a class file named epfl.sweng.tests.AllTests (a template is provided in the archive you downloaded); you are free to split your test suite into multiple files, but running epfl.sweng.tests.AllTests should run all tests
- Do not modify any source file except those that you place in the epfl.sweng.tests package; we will run your test suite against our vanilla HW#3 solution
- The test suites themselves should not be included in the coverage measurements.
- Ensure your test suite verifies the functionality of the program (e.g., use assertion checks in the form of Assert.assertTrue(myAdd(1,2)==3);
instead of merely a call to myAdd). We will run your test suite against buggy versions of the solution, and they should fail accordingly, or else the corresponding test will be invalidated.
We will compute the number of points as follows:
# points = 90 * (0.1 * fc/100 + 0.4 * sc/98 + 0.5 * bc/94)
- fc is the function coverage (in percentage) obtained by your test suite
- sc is statement coverage (in percentage) obtained by your test suite
- bc is branch coverage (in percentage) obtained by your test suite
For example, for fc=100
, you will get 72 points.
Points above 90 will count as a bonus.
Our solution has several bugs in it (in addition to the one listed below in Part 2). If you find a bug while writing your tests, send your report to the sweng staff list along with a proposed fix. If we confirm the bug and the solution, you will receive a 5 points bonus. For a given bug, only the first reporter with a correct solution will get the points.
Part 2. Fixing Bugs using Test Information (10 points)
In this part of the homework, you will see that 100% code coverage and 100% branch coverage are not sufficient to guarantee freedom from bugs.
The TA-developed solution that you used in part 1 contains one bug in the computePredecessors() method of the epfl.sweng.CourseScheduler.LectureScheduler class.
This bug is hard to trigger, even with 100% coverage.
To understand the bug, you must understand the Weighted Interval Scheduling algorithm implemented by the LectureScheduler class.
You can find a detailed explanation here.
For this exercise, do the following:
- Provide a test case that returns success and achieves 100% statement coverage of the
computePredecessors() method in the LectureScheduler class
- Provide another test case that exhibits the bug, i.e., returns failure if the bug is present and success otherwise
- Fix the bug in the provided solution
- Demonstrate you fixed the bug by re-running the two test cases; all assertion checks must succeed.
You must stick to the following rules:
- The test case that succeeds and the test case that exhibits the bug must be placed in epfl.sweng.tests.TestBug (a template is provided in the archive you downloaded).
- You must annotate the location of the bug with //BUG: <explanation>.
Deliverables for Homework #4
Hand in your assignment by committing all the required files to your SVN account. Below, SVNAccount stands for the actual path to your SVN account (e.g., https://svn.epfl.ch/svn/sweng2010-candea):
- Place your answers in a file named SVNAccount/trunk/assignment4/answers.txt.
You must use the provided answers.txt template this time.
- Specify which code base you used for your test suite in answers.txt. Replace Revision # with Revision 3 or Revision 4, depending on the revision of the code base you used.
- Provide your Java source files in a folder named SVNAccount/trunk/assignment4, which includes also the sources of the test classes. Failing to follow this file structure will result in losing all the points for this assignment. There will be no exceptions this time.
- Your code must successfully compile on the BC machines, by running bash -c "/usr/bin/find -name '*.java' | xargs javac -cp 'C:\eclipse\plugins\org.junit4_4.5.0.v20090824\junit.jar'". This command will search for all the Java files in the assignment directory and compile them with javac. If your files do not compile, you will lose all of the points for the code writing exercises that are affected by the compilation errors.
- You must write how much time (in hours) you spent solving each exercise in a file named SVNAccount/trunk/assignment4/timeSpent.txt file. You must use the provided template this time.
- You are strongly encouraged to commit your homework deliverables from the BC machines, since occasional network connectivity problems may prevent you from submitting from other locations.