Correctness

Domain Partitioning

Essential black-box techniques for generating tests for functional testing.

• Equivalence class partitioning

• Boundary value analysis

Test Generation Techniques

Useful during functional testing where objective is to test whether or not an application (module, unit, subsystem, or system) correctly implements the functionality of given requirements\/specifications

Documents

• Test Plan: Describe scope, approach, resources, test schedule, items to be tested, deliverables, responsibilities, approvals needed. Could be used at the system test level or at lower levels.

• Test design spec: Identifies a subset of features to be tested and identifies the test cases to test the features in this subset.

• Test case spec: Lists inputs, expected outputs, features to be tested by this test case, and any other special requirements e.g. setting of environment variables and test procedures. Dependencies with other test cases are specified here. Each test case has a unique ID for reference in other documents.

• Test procedure spec: Describe the procedure for executing a test case.

• Test transmittal report: Identifies the test items being provided for testing, e.g. a database.

• Test log: A log observations during the execution of a test.

• Test incident report: Document any special event that is

  recommended for further investigation.

• Test summary: Summarize the results of testing activities and provide an evaluation

Test Selection Problem

• Select a subset T of tests such that the execution of P against each element of T will reveal all errors in P

• There does not exist any algorithm to construct such a test set.

  • There are heuristics and model based methods that can be used _to generate tests _that will reveal certain type of faults

• The Challenge: construct a test set T, a subset of D, that will reveal as many errors in P as possible
  • Difficult b\/c size and complexity of the input domain of P
    • Exhaustive testing: large size of the input domain prevents a tester from exhaustively testing the program under test against all possible inputs (testing program against every element in the input domain)
      • complexity makes it difficult to select individual tests

• Solution: Equivalence Partitioning

Equivalence partitioning

allows a tester to subdivide the input domain into a small number of sub-domains, say N>1

• sub-domains by definition are disjoint

• The four subsets (equivalence classes) shown in (a) constitute a partition of the input domain (b) are not

  • Equivalence classes are created assuming that the program under test exhibits the same behavior on all elements, i.e. tests, within a class.

    • Allows the tester to select exactly one test from each equivalence class − resulting in a test suite of exactly N tests

• The entire set of inputs of an application can be divided into at least two subsets:

  1. All the expected (legal) inputs (E)

  2. All unexpected (illegal) inputs (U)

• Each of the two subsets, can be further subdivided into subsets on which the application is required to behave differently e.g. E1, E2, E3, and U1, U2

• Equivalence class partitioning selects tests that target any faults in the application that cause it to behave incorrectly when the input is in either of the two classes or their subsets.

Example

Consider an application A that takes an integer denoted by age as input.

• Suppose that the only legal values of age are in the range [1..120].

• The set of input values is now divided into

  • a set E containing all integers in the range [1..120] and

  • a set U containing the remaining integers.

• Further, assume that the application is required to process all values in the range [1..61] in accordance with requirement R1 and those in the range [62..120] according to requirement R2

  • Thus, E is further subdivided into two regions depending on the expected behavior.

• Similarly, it is expected that all invalid inputs

  • less than 1 are to be treated in accordance with requirement R3

  • while all greater than 120 are to be treated in accordance with requirement R4

This leads to a subdivision of U into two categories

Tests selected using the equivalence partitioning technique target faults under test with respect to inputs in any of the four regions, i.e.,

• two regions containing expected inputs

• two regions containing the unexpected inputs.

It is expected that any single test selected from

• the range [1..61] will reveal any fault with respect to R1.

• the region [62..120] will reveal any fault with respect to R2.

• A similar expectation applies to the two regions containing the unexpected inputs.

Equivalence class testing

identify test cases by using one element from each equivalence class.

• Equivalence classes chosen carefully\/wisely: greatly reduces the potential redundancy among test cases.

• The key point: the choice of the equivalence relation that determines the classes (partitions)

Motivations: Gives Sense of complete testing & Avoids redundancy

Equivalence classes: form a partition of a set, a collection of mutually disjoint subsets whose union is the entire set

Implications:

1. The entire set represented provides a form of completeness

2. Disjointness assures a form of non-redundancy

Selection:

• Based _only the requirements\/specification (_no dependence on knowledge of code,)

• Needs knowledge of input domain that usually goes beyond what an interface design specification provides familiarity with specifications

  • must understand how inputs are mutually dependent

Categories: strong, weak

Example: