Software Testing

Ensuring correctness, reliability, and performance through structured testing techniques.

Foundations of Software Testing

Core concepts, testing types, and execution methods

What is Software Testing?

Software testing is the process of identifying the correctness of software by evaluating its components to find bugs, errors, or defects. It evaluates attributes like reliability, scalability, portability, re-usability, and usability.

It is a critical element of software quality assurance. In fact, testing is viewed as a destructive process where engineers create test cases specifically intended to "defeat" the built software.

The 7 Principles of Testing

  • Complete testing is not possible: Exhaustive testing is impossible; risk assessment is used instead.
  • Defect clustering: ~80% of defects are found in 20% of the modules.
  • Pesticide Paradox: Repetitive use of the same test cases stops finding new defects; they must be reviewed and revised.
  • Testing shows presence of defects: It proves defects exist, but cannot guarantee 100% absence of errors.
  • Cannot use only existing mechanisms: Testers must constantly improve methods.
  • Absence of error fallacy: 99% bug-free software is still useless if it doesn't satisfy user requirements.
  • Early Testing: Testing should start early in the SDLC (e.g., requirement gathering) to reduce fix costs.
  • Context dependent: Testing approaches change entirely based on the application type.

Manual vs. Automation Testing

Parameter Manual Testing Automation Testing
Human Interaction Requires human interaction for test execution. Human interaction is not required; tools run the test cases.
Requirements Requires skilled employees, more time, and high costs. Requires automation tools. Saves time, cost, and manpower once recorded.
Application to Test Any application can be tested manually first (e.g., ad-hoc testing). Used for applications with stable requirements (mainly Regression Testing).
Execution Repetitive execution of the same test suite becomes boring and error-prone. Easily handles the boring part of running the exact same test cases repetitively.

Static Testing vs. Dynamic Testing

Static Testing

Software application is tested without code execution. It is performed before compilation to prevent defects.

Techniques Used:

  • Informal Reviews: Reviewing documents without a strict process.
  • Technical Reviews: Peers check technical specs (Test Plans, Strategy) for discrepancies.
  • Walkthrough: The author explains the product to the team.
  • Inspection: A formal review led by a trained moderator using a checklist.
  • Static Code Review: Systematic review of source code syntax and standards without executing it (White Box).

Dynamic Testing

Testing is done by executing the program. It is performed after compilation to find and fix defects.

Techniques Used:

  • Unit Testing: Testing individual source code modules by developers.
  • Integration Testing: Testing how grouped modules interact with each other.
  • System Testing: Checking the whole integrated system against the requirement specification document (functionality, security, portability).
Static Testing Dynamic Testing
We check the code or application without executing the code. We check the code/application by executing the code.
It is a Verification Process used to prevent defects. It is a Validation Process used to find and fix defects.
A more cost-effective process performed before compilation. A less cost-effective process done only after executables are prepared.
Involves activities like Code Review, Walkthroughs, and Inspections. Involves Functional and Non-functional testing (Unit, Integration, System, UAT).
Involves the checklist and process followed by the test engineer. Requires test cases for the execution of the code.

Levels of Software Testing

The structured progression from individual components to the final product

Software development spirals inward from system engineering down to coding, but software testing spirals outward. The levels of testing involve different methodologies used as the scope of the software expands:

1

Unit Testing

Tests the individual, isolated component. Targets the code itself.

2

Integration Testing

Tests the integrated components together. Targets the software design.

3

System Testing

Tests the entire computer-based system. Validates against system engineering requirements.

4

Acceptance Testing

Tests the final system for delivery. Validates actual user needs.

1. Unit Testing

Isolating and verifying the smallest building blocks of code

A "unit" is the smallest testable part of a software system, which may include individual code files, classes, methods, functions, procedures, or objects. Unit testing validates these small building blocks much before testing an integrated large module.

  • Who & When: Done during the development (coding) phase by the developers themselves.
  • Methodology: It is a strategy that utilizes the white-box method.
  • Tools: Automated unit testing is highly beneficial. Examples include JUNIT (for Java) and XUnit (a general framework for C#, ASP, C++, Delphi, and Python).

What gets evaluated during a Unit Test?

1. Module Interface Test

Checks whether the information is properly flowing into the program unit and properly happening out of it.

2. Local Data Structures

Tested to inquire if the local data within the module is stored properly.

3. Boundary Conditions

Because software often fails at boundaries, conditions are tested to ensure the program works properly at its limits.

4. Independent Paths

All independent paths are tested to verify they properly execute their task and terminate at the end of the program.

5. Error Handling Paths

Tested to review if errors are caught and handled properly by the code.

Auxiliary Code: Drivers and Stubs

Because a unit does not exist in isolation, developers must develop auxiliary software (drivers and stubs) for each unit test to simulate the rest of the application.

The Driver

A "main program" that accepts test case data, passes that data to the component being tested, and prints the relevant results.

The Stub

A "dummy subprogram" that uses the subordinate module's interface to replace modules that are called by the component being tested.

DRIVER
(Calls the Unit)
MODULE / UNIT
STUB
(Called by the Unit)

Advantages

  • Developers can look at unit tests to easily learn the unit API and what functionality is provided.
  • Allows programmers to safely refactor code later (Regression testing) and ensure the module still works correctly.
  • When a change causes a fault, it can be quickly identified and fixed because test cases are written for all functions.
  • The modular nature allows testing parts of the project without waiting for others to be completed.

Disadvantages

  • Cannot catch every error; it is simply not possible to evaluate all execution paths, even in trivial programs.
  • Because it focuses strictly on a unit of code, it cannot catch integration errors.
  • It completely misses broad system-level errors.

2. Integration Testing

Exposing defects in the interaction and communication between grouped modules

Integration testing is the second level of the software testing process, occurring directly after unit testing. It is defined as a type of testing where software modules are integrated logically and tested as a group.

  • The Purpose: To expose defects in the interaction between these software modules when they are integrated. It focuses on checking data communication amongst modules.
  • Application: It is especially relevant to client/server and distributed systems where modules represent individual applications on a network.
  • Alternate Names: It is also referred to as 'I & T' (Integration and Testing), 'String Testing', and 'Thread Testing'.

A. Incremental Integration Testing

In the incremental approach, logically related modules are added one by one in ascending or descending order (e.g., Login → Home → Search → Add cart). The process continues until the successful testing of all modules.

Top-Down

Uses Stubs

Testing takes place from top to bottom following the control flow. The higher-level modules are tested first. Stubs are used to simulate missing lower-level modules.

Advantages:
  • Fault localization is easier.
  • Possibility to obtain an early prototype.
  • Critical modules are tested on priority, so major design flaws can be detected and fixed early.
Disadvantages:
  • Needs many Stubs.
  • Modules at a lower level are tested inadequately.

Bottom-Up

Uses Drivers

The lower-level modules are tested first, and then these tested modules are used to facilitate the testing of higher-level modules.

Advantages:
  • Fault localization is easier.
  • No time is wasted waiting for all modules to be developed (unlike Big-bang).
Disadvantages:
  • Critical modules at the top level, which control the flow of the application, are tested last and may be prone to defects.
  • An early prototype is not possible.

Sandwich / Hybrid

Uses Stubs & Drivers

Top-level modules are tested with lower-level modules at the same time, merging together as a system. It is a combination of Top-down and Bottom-up.

Advantages:
  • Provides features of both Bottom Up and Top Down methods.
  • Provides complete testing of all modules.
  • It is the most time-reducing method.
Disadvantages:
  • A highly complicated method.
  • Needs a higher level of concentration as the process is carried out in both directions simultaneously.

B. Non-Incremental (Big Bang) Testing

In this approach, testing is done via the integration of all modules at once, and this combined set is considered as a single entity. This is typically used when data flow is very complex and finding parent/child relationships is difficult.

Advantages

  • It is convenient for small size software systems.

Disadvantages

  • Fault Localization is difficult: Finding the source of a bug is a major problem because everything is integrated at once.
  • Time Constraints: Testing can only commence after "all" modules are designed, leaving the testing team with very little time for execution.
  • Missed Interfaces: Given the sheer number of interfaces tested simultaneously, internally linked interfaces and small modules can easily be missed.
  • Critical Modules Ignored: High-risk critical modules and peripheral UI modules are not isolated and tested on priority.

3. Validation Testing

Ensuring the software fulfills its intended use and client needs

Validation testing determines if the existing system complies with the system requirements and performs the dedicated functions for which it was designed. It evaluates the software during or at the end of the development process to ensure it satisfies specified business requirements.

The Ultimate Goal

Validation testing answers a very specific question: "Are we building the right product?". It is important because it ensures customer satisfaction, builds confidence in the product, and ultimately leads to software acceptance from the end-user.

Verification vs. Validation

Validation is often confused with Verification. Verification happens before validation and ensures the product is being built correctly.

Verification (Static) Validation (Dynamic)
Goal: "Are we developing the product right?" Goal: "Are we building the right product?"
Checks the work-products (not the final product) to see if they meet specified requirements. Checks the software during or at the end of development to see if it follows business requirements.
No execution of code happens (Static testing). Execution of code happens (Dynamic testing).
Includes methods like Inspections, Reviews, and Walkthroughs. Includes functional testing, system testing, integration, and UAT.
Executed early in the development phase by the Quality Assurance team. Executed later by the Testing team to find bugs missed during verification.

4. System Testing

Evaluating the end-to-end, fully integrated software product

System testing validates the complete and fully integrated software product. Because software is usually only one element of a larger computer-based system, it must be tested to see how it interfaces with other software and hardware systems.

  • It checks how components interact with one another and with external peripherals.
  • It verifies every input in the application checks for desired outputs (End-to-End scenario).
  • It tests the user's overall experience with the application.

Types of System Testing

Usability Testing

Also known as User Experience testing. A group of end-users check if the software is user-friendly (e.g., ensuring icons are not confusing). It is non-functional testing.

Performance Testing

Carried out to test the speed, scalability, stability, and reliability of the software product.

Load & Stress Testing

Load: Determines behavior under extreme load. Stress: Checks the robustness of the system under varying, unexpected loads.

Regression Testing

Re-executing old test cases to ensure that new code additions or bug fixes haven't disturbed existing, working functionality.

Recovery Testing

Checks if the system recovers itself after crashing (e.g., power failure). The system should identify a correct point and perform a rollback.

Migration Testing

Ensures the software can be moved from an older infrastructure to a current infrastructure without data compatibility problems.

Functional Testing

A black-box testing type that checks every function works exactly as per user requirements. Actual results are compared against expected results using the Requirement Specification document.

Out of Syllabus / Supplementary

Acceptance Testing (UAT)

Final validation by the end-user before production

User Acceptance Testing (UAT) is performed by the end-user or client to verify and accept the software system before moving it to the production environment. It is conducted in the final phase of testing, after functional, integration, and system testing are complete.

Why is it needed?

  • Developers code software based on their "own" understanding of the requirements document, which might not perfectly align with what the client actually needs.
  • Requirement changes made during the project course may not have been communicated effectively to the developers.
  • It validates the end-to-end business flow, minimizing issues arising from production.

Note: UAT is a kind of black-box testing carried out in a separate testing environment with production-like data, and it does not focus on cosmetic errors or spelling mistakes.

Key Acceptance Types: Alpha vs. Beta Testing

Acceptance testing encompasses several variations, including Operational Acceptance Testing (OAT), Contract Acceptance Testing (CAT), and Business Acceptance Testing (BAT). The two most prominent methods are Alpha and Beta testing.

Alpha Testing Beta Testing
Conducted at the developer's site in a controlled/lab environment. Conducted at client locations or end-user sites without a lab environment.
Performed by a team of highly skilled testers (internal employees) or a representative group of users. Performed by actual clients or real end-users (not organization employees).
The developer is present, "looking over the shoulder" of the users. The developer is generally not present; it is a "live application" of the software.
Involves both white-box and black-box techniques. Uses only black-box testing techniques.
Critical issues or fixes can be identified and implemented by developers immediately. Feedback and issues collected are typically implemented in future versions of the product.
Performed near the end of software development, before launching into the market. The final test before shipping the product to customers, done at the time of software marketing.

White-Box Testing: Basis Path Testing (Part 1)

Structural testing, control flow graphs, cyclomatic complexity, and connection matrices

1. What is White-Box Testing?

White-Box Testing is a structural testing technique where the internal structure, design, and source code of the software are tested to verify input-output flow and improve software design, usability, and security.

The "See-Through Box" Concept

The name symbolizes being able to look through the software's outer shell into its inner workings. Because the code is completely visible to the tester, it is also called Clear box, Open box, Transparent box, Glass box, or Code-based testing.

What Does it Directly Test?

  • Internal security holes and vulnerabilities.
  • Broken or poorly structured paths in coding logic.
  • The flow of specific inputs line-by-line through the code.
  • Conditional loops executed at operational bounds.
  • Testing of each individual statement, object, and function.
  • Internal data structures to ensure validity.

Applicable Testing Levels

  • Unit Testing: For testing code paths within an isolated unit.
  • Integration Testing: For testing paths connecting different units.
  • System Testing: For testing execution paths between major subsystems.
Advantages
  • Optimizes code by exposing hidden defects and dead code.
  • Test cases can be easily automated.
  • Very thorough, as all execution paths are covered.
  • Can begin early in SDLC even before the GUI is created.
Disadvantages
  • Complex, expensive, and time-consuming for large programs.
  • Requires expert resources with thorough programming knowledge.
  • Tests executed casually by developers can miss edge cases.

2. Basis Path Testing

A program contains multiple entry points, exit points, conditions, and loops. Testing every single possible path is impossible and redundant.

Basis Path Testing is a white-box testing method where test cases are defined based on logical paths through the program. It is a hybrid method combining statement coverage and branch coverage. Its objective is to define the number of independent paths so that the exact number of required test cases can be determined to achieve maximum path coverage with the fewest tests.

The 5 Core Steps of Basis Path Testing:

  1. Construct the Control Flow Graph (CFG).
  2. Compute the Cyclomatic Complexity V(G) of the graph.
  3. Identify the Independent Paths (the Basis Set).
  4. Design Test Cases representing each independent path.
  5. Generate Graph Matrices & Connection Matrices to verify the metrics.

Step 1: Control Flow Graph (CFG) & Terminologies

A Control Flow Graph (flow graph) is a directed graph G = (V, E) where V is the number of vertices/nodes and E is the number of edges/links.

Node (Vertex)

Represents one or more procedural statements or code lines.

Predicate Node

A node containing a condition that leads to two or more branches.

Edge (Link)

Directed arrows connecting nodes, showing flow of control. Must terminate at a node.

Region

An area bounded by edges and nodes. The area outside the entire graph always counts as 1 region!

Decision Node

A node with more than one arrow leaving it (branches out).

Junction Node

A node with more than one arrow entering it (branches merge).

Standard Program Construct Notations
Sequence 1 2
If-Then-Else 1 2 3 4
Do-While 1 2 3
Switch-Case 1 n+1

Step 2: Calculating Cyclomatic Complexity V(G)

Cyclomatic complexity V(G) is a quantitative measure of the logical complexity of a program. It defines the maximum number of independent paths in the basis set.

It can be calculated using 3 separate formulas (all 3 always give the exact same answer):

Formula 1: Edges & Nodes
V(G) = e - n + 2*P
  • e = Number of edges (arrows)
  • n = Number of nodes (vertices)
  • P = Connected components (P = 1 for a single program graph)
Formula 2: Decision Nodes
V(G) = d + P
  • d = Number of decision/predicate nodes
  • P = Number of connected graphs (usually 1)
  • Rule: If a node has k outgoing arrows, count it as d = k - 1!
Formula 3: Regions
V(G) = Number of Regions
  • Count enclosed internal regions.
  • Add +1 for the open region surrounding the outside of the graph.
Example: Comparing the 3 Formulas on a Branch (Diamond Graph)
Node 1 A C Node 2 Node 3 B D Node 4 Region 1 Region 2 (Outer)

Parameters: Nodes (n) = 4, Edges (e) = 4 (A, B, C, D), Connected graphs (P) = 1

Formula 1: V(G) = 4 - 4 + 2*(1) = 2

Formula 2: Only Node 1 has two outgoing edges (d = 1). V(G) = 1 + 1 = 2

Formula 3: Region 1 (inner) + Region 2 (outer) = 2

All three methods confirm V(G) = 2!

Properties of Cyclomatic Complexity:

1. V(G) is the maximum number of independent paths in the graph.

2. V(G) ≥ 1.

3. If V(G) = 1, the program has exactly one linear path with no decisions.

4. Good practice guideline: Keep V(G) ≤ 10 to minimize testing complexity and risk.

Uses of Cyclomatic Complexity
  • Determines Independent Path Executions: Helps developers and testers systematically identify the exact number of execution paths that must be tested.
  • Assures Complete Path Coverage: Allows developers to confirm that all procedural paths have been executed at least once.
  • Targets Uncovered Code: Helps testers focus their attention on untested or uncovered paths instead of writing redundant test cases.
  • Improves Code Coverage: Directly enhances the overall code and logic coverage metrics in software engineering.
  • Evaluates Module Risk: Measures the logical complexity of the code, serving as a reliable metric to gauge the failure risk of a given module.
  • Early Risk Reduction: Utilizing these complexity metrics early in the software life cycle prevents complicated, high-risk code from reaching production.

Steps 3 & 4: Independent Paths & Test Case Design

An Independent Path is any path through the flow graph that introduces at least one new edge that has not been traversed before the path is defined. The cyclomatic complexity V(G) gives the exact number of independent paths present in the graph.

Control Flow Graph (7 Nodes, 9 Edges) 1 2 3 4 5 6 7
Exam Problem Solving: Sorting Loop Flow Graph
// Given Code:
i = 0;
n = 4;
1: while (i < n-1) do
    j = i + 1;
2:   while (j < n) do
3:     if A[i] < A[j] then
4:       swap(A[i], A[j]);
      end do;
5:     j = j + 1;
    end do;
6:   i = i + 1;
    end do;
7: exit;
Calculate V(G):
• Nodes = 7, Edges = 9
• Formula: V(G) = 9 - 7 + 2*(1) = 4
• Predicate nodes = Nodes 1, 2, 3 (each has 2 exit branches): 3 + 1 = 4
The Basis Set (4 Independent Paths):
  • Path 1: 1, 7
  • Path 2: 1, 2, 6, 1, 7
  • Path 3: 1, 2, 3, 4, 5, 2, 6, 1, 7
  • Path 4: 1, 2, 3, 5, 2, 6, 1, 7

* Note: Independent paths are not unique. If V(G) = 4, another tester might pick a different combination of routes, but there will always be exactly 4 independent paths needed to cover the basis set.

Step 5: Graph Matrices & Connection Matrix

A graph matrix is a square matrix whose rows and columns equal the number of nodes in the flow graph. Each entry represents an edge connection from a row node to a column node. Numbers denote nodes, and letter symbols denote edges.

When the letter symbols are replaced by link weights of 1 (connection exists) or left blank (no connection), it is called a Connection Matrix.

Flow Graph a b c d f e g 1 3 4 5 2
Graph Matrix (5x5)
Node 1 2 3 4 5
1 a
2
3 d b
4 c f
5 g e

Letters represent directed connections from row node to column node.

Calculating V(G) from Matrix:
  1. Count connections (weight = 1) in each row:
    • Row 1 = 1 entry ('a')
    • Row 2 = 0 entries
    • Row 3 = 2 entries ('d', 'b')
    • Row 4 = 2 entries ('c', 'f')
    • Row 5 = 2 entries ('g', 'e')
  2. Subtract 1 from each non-zero row:

    (1-1) + (2-1) + (2-1) + (2-1) = 0 + 1 + 1 + 1 = 3

  3. Add 1 to that sum:

    3 + 1 = 4

  4. Cyclomatic Complexity V(G) = 4!

Key Benefits & Applications of Basis Path Testing:

• Maximum Code Coverage: Aims to achieve maximum logic coverage instead of arbitrary path testing.

• Maintenance Testing: Used when modifying software to verify newly added code paths without breaking existing paths.

• Proportional Effort: Testing effort is directly proportional to the software's cyclomatic complexity.

3. Control Structure Testing

While Basis Path Testing guarantees that every statement and branch is touched at least once, it does not thoroughly test complex boolean logic or repeated loop iterations.

Control Structure Testing is used to increase test coverage by systematically exercising the various logical control structures present in a program. It is divided into four distinct testing techniques:

1. Branch Testing

Tests true and false decision outcomes.

2. Condition Testing

Validates individual logical operators and relational expressions.

3. Data Flow Testing

Tracks where variables are defined and used across statements.

4. Loop Testing

Tests loop boundaries, iterations, and nesting.

Technique 1: Branch Testing (Decision Testing)

In Branch Testing, test cases are designed so that for every decision point in the program, each possible branch (True and False) is executed at least once.

Key Characteristic: Compound Conditionals

Branch testing treats compound conditionals as a single consolidated decision statement. It counts only the physical branches taken out of the decision, ignoring the implicit hidden paths inside compound boolean operators.

Branch Calculation Examples:

Simple Decision

IF (a equals b) THEN
  statement 1
ELSE
  statement 2
END IF

Branches = 2

Compound Decision

IF (a equals b AND c < d) THEN
  statement 1
ELSE
  statement 2
END IF

Branches = 2
(Treated as single decision)

Nested Decision

IF (a equals b) THEN stmt 1
ELSE
  IF (c equals d) THEN stmt 2
  ELSE stmt 3
  END IF
END IF

Branches = 4

Obvious vs. Subtle Decisions in Source Code:

• Obvious decisions: if, for, while, and switch statements.

• Subtle decisions: return boolean expressions, ternary expressions (? :), and try-catch blocks.

* Note: Testers do not need to write test cases for low-level system errors like IOException or OutOfMemoryException.

Technique 2: Condition Testing

Condition testing is a test case design method that focuses specifically on exercising the logical conditions contained within a program module to ensure they are free from errors.

Types of Logical Conditions Tested:

  • Relational Expression: E1 op E2 (where E1 and E2 are arithmetic expressions and op is a comparison operator like <, >, ==).
  • Simple Condition: A relational expression or a condition preceded by a NOT operator (~E1).
  • Compound Condition: Two or more simple conditions combined with Boolean operators and parentheses, such as (E1 & E2) | (E2 & E3).
  • Boolean Expression: Operands combined using boolean operators (AND, OR, NOT), such as A | B.
Common Errors Detected:
  • Boolean operator errors (e.g., using OR instead of AND).
  • Boolean variable errors.
  • Missing or misplaced parentheses in Boolean expressions.
  • Relational operator errors (e.g., using > instead of >=).
  • Arithmetic operation errors inside conditions.
Multiple-Condition Coverage Rule

For a compound condition C, testing requires that the true and false branches of C, as well as every simple condition inside C, be executed at least once.

Multiple-Condition Testing requires exercising all True-False combinations of simple conditions at least once, ensuring that all statements, branches, and conditions are fully covered.

Technique 3: Data Flow Testing

Data flow testing selects test paths according to the locations of definitions and uses of variables in the program. It inspects the lifecycle of data: where a variable is declared/assigned (definition) and where it is subsequently read or referenced (use).

Practical Usage Guidelines

It is unrealistic and impractical to use data flow testing across an entire large software system because the number of variable combinations is astronomical.

Targeted Application: It is used in a targeted fashion for suspect areas of software, particularly modules with deeply nested if statements and complex loop structures.

Technique 4: Loop Testing

Loops are fundamental to software algorithms and require targeted verification. Loop testing is a white-box technique that focuses entirely on the validity of loop constructs. Loops are categorized into four classes: Simple, Nested, Concatenated, and Unstructured.

A. Simple Loops

Where n is the maximum allowable number of iterations through the loop, the following 7 test cases must be applied:

  • 1. Skip the entire loop (0 iterations)
  • 2. Traverse the loop only once (1 iteration)
  • 3. Traverse the loop two times (2 iterations)
  • 4. Make p passes through the loop (where p < n)
  • 5. Traverse the loop n - 1 times (just below maximum)
  • 6. Traverse the loop n times (maximum boundary)
  • 7. Traverse the loop n + 1 times (boundary error check)
Process Cond?
B. Nested Loops

Loops within loops. If tested blindly, the number of test cases increases geometrically.

Standard Step-by-Step Procedure:

  1. Start with the innermost loop.
  2. Set all other outer loops to their minimum parameter values.
  3. Conduct simple loop testing on the innermost loop.
  4. Work outwards, holding outer loops to minimums and keeping already-tested inner loops at typical values.
  5. Continue until all loops have been tested.
C. Concatenated Loops

Two or more loops executing in sequence along the execution path.

  • Independent Loops: If the loops are not dependent on each other, test each using the simple loop approach.
  • Interdependent Loops: If one loop's counter or output governs the next loop, treat them as nested loops.
D. Unstructured Loops

Loops with jumps, multiple entries, or arbitrary branches cutting across boundaries.

Testing Rule: These should NOT be tested as-is!

Whenever encountered, redesign the code immediately to use standard structured programming constructs.

Trade-offs of Control Structure Testing

Advantages
  • Provides thorough testing of program logic.
  • Finds defects at an early stage.
  • Eliminates dead or unreachable code.
  • Not time-consuming as it is mostly automated.
Disadvantages
  • Requires complete source code knowledge to perform tests.
  • Requires extensive training in the testing tools used.
  • Can become expensive for complex software.

Black-Box Testing: Graph-Based Testing

Behavioral verification, input-output validation, and graph modeling

1. What is Black-Box Testing?

Black-Box Testing is a testing technique that examines the software from the user's perspective without any knowledge of internal code, server logic, or implementation details. Because it focuses exclusively on functional behavior rather than code structure, it is also known as Behavioral Testing.

The Black-Box Concept

The software is treated as an opaque black box. The tester provides an Input based on the customer requirements and verifies whether the system produces the expected Output without caring how the code calculates it.

Input → BLACK BOX → Output

Primary Errors Detected:

  • Incorrect or missing functions.
  • Interface errors and UI usability flaws.
  • Data structure or external database access errors.
  • Performance, response-time, or initialization errors.
  • Abrupt application crashes or termination issues.

Major Types of Black-Box Testing:

  • Functional Testing: Checks how correctly the system performs its required functions.
  • Non-Functional Testing: Evaluates readiness based on non-functional criteria like performance, scalability, and usability.
  • Regression Testing: Executed after code fixes or upgrades to ensure existing working features remain unaffected.

Standard Process for Black-Box Testing:

  1. Examine requirements and system specifications.
  2. Select valid inputs (positive test cases) to confirm correct processing.
  3. Select invalid inputs (negative test cases) to verify defect handling.
  4. Determine expected outputs for every input.
  5. Construct test cases, execute them, and compare actual outputs against expected outputs.
  6. Log defects, report them for fixes, and re-test.

2. Graph-Based Testing (State-Based Testing)

Graph-Based Testing (also called State-Based Testing) is a black-box method that provides a formal framework for model-based testing. It models the relationships (links) among software program objects (nodes) as a directed graph, and test cases are designed to traverse the entire graph.

The 4 Graph Modeling Paradigms

1. Transaction Flow Modeling

Nodes represent procedural steps in a user transaction, and links represent logical connections between steps that require validation.

2. Finite State Modeling

Nodes represent user-observable states of the software, and links represent events or transitions that move the system between states.

3. Data Flow Modeling

Nodes represent data objects, and links represent functional transformations applied from one data object to another.

4. Timing Modeling

Nodes represent program objects, links represent sequential connections, and link weights represent required execution times.

Step-by-Step Procedure to Design Graph Test Cases:

  1. Understand the System: Analyze the domain and user functional requirements.
  2. Identify Entities: Identify system states, allowed inputs, and conditional guards.
  3. Build Graph Model: Construct a directed state or transaction graph model.
  4. Verify Graph: Ensure the modeled graph reflects system requirements accurately in all details.
  5. Identify Test Paths: Generate a sequence of test actions covering all nodes and links.
  6. Derive Test Data: Formulate specific input data to physically execute those paths during testing.

Example Model: Leave Application Approval System

The following transaction flow graph represents an employee leave approval scenario with multiple decision paths:

Employee Desires Leave Submit Form HR Verify Eligibility Eligible Ineligible Manager Feasibility Feasible Not Feasible Approved Rejected

Test cases derived from this model must cover all 3 operational paths: (1) Approved path, (2) HR Ineligible rejection path, and (3) Manager non-feasible rejection path.

Black-Box Testing: Equivalence Class Testing

Minimizing test cases while maximizing coverage by grouping identical program behaviors

1. What is Equivalence Class Partitioning (ECP)?

Equivalence Class Testing (or Partitioning) is a foundational black-box testing method used to reduce the infinite number of possible input test cases down to an optimal, manageable level while maintaining high test coverage.

The Core Principle

The entire input domain of a program is divided into distinct partitions (classes) of valid and invalid values. It is assumed that all values inside a single partition produce the exact same behavioral outcome from the software. Therefore, testing just one representative value from a partition is equivalent to testing every single value in that partition.

Key Objectives:

  • Reduces a massive set of potential test inputs into a small, highly effective set of test cases.
  • Guarantees that each defined partition is exercised at least once.
  • Evaluates both valid states (positive testing) and invalid states (negative testing).

Why Not Test Everything?

If a field accepts any integer from 1 to 100, running 100 positive tests and thousands of negative tests wastes time and resources. If the code works for input 25, it almost certainly works identically for 26, 27, and 50. ECP groups them logically so you test only one representative value per group.

2. Guidelines for Defining Equivalence Classes

Standard testing rules dictate how many valid and invalid classes you must construct based on the type of input data requested:

1. Range Condition 1 Valid, 2 Invalid

If input is specified as a range of values (e.g., 10 to 50):

  • 1 Valid Class: Values within the range (10 ≤ x ≤ 50).
  • Invalid Class 1: Values below the minimum (x < 10).
  • Invalid Class 2: Values above the maximum (x > 50).
2. Specific Value Condition 1 Valid, 2 Invalid

If input specifies an exact expected value (e.g., must enter exactly 100):

  • 1 Valid Class: The specific exact value itself.
  • Invalid Class 1: Values less than the specified value.
  • Invalid Class 2: Values greater than the specified value.
3. Member of a Set 1 Valid, 1 Invalid

If input specifies a defined discrete set of values (e.g., Vehicle: Car, Bus, Bike):

  • 1 Valid Class: Any member that belongs to the permitted set.
  • 1 Invalid Class: Any value outside the defined set (e.g., Airplane).
4. Boolean Condition 1 Valid, 1 Invalid

If input specifies a Boolean flag or binary state:

  • 1 Valid Class: The true/active condition state.
  • 1 Invalid Class: The false/inactive condition state.

3. Step-by-Step Practical Examples

Example 1: User Registration "AGE" Textbox

Range Input

Scenario: A software application form contains an "AGE" text box that strictly accepts integer values between 18 and 60. Anything outside this range must trigger a validation error.

Invalid Partition 1 Valid Partition Invalid Partition 2
≤ 17 18 to 60 ≥ 61
Selected Test Value 12 Selected Test Value 34 Selected Test Value 75
Outcome: Validation Error Outcome: Accepted Outcome: Validation Error

Instead of testing ages 0 through 120, choosing just three values (e.g., 12, 34, and 75) is sufficient to verify the boundaries and the core acceptance logic.

Example 2: Password Length Validation

Range Input

Scenario: A system requires user passwords to be between 6 and 12 characters long.

Invalid Partition (Short)

0 to 5 characters

"Abc"

Length: 3 (Rejected)

Valid Partition

6 to 12 characters

"Secure12"

Length: 8 (Accepted)

Invalid Partition (Long)

13+ characters

"WayTooLongPass123"

Length: 17 (Rejected)

Black-Box Testing: Boundary Value Analysis (BVA)

Catching defects where they hide most often: at the edges of equivalence partitions

1. What is Boundary Value Analysis?

Boundary Value Analysis (BVA) is a test design technique that complements Equivalence Class Partitioning (ECP). While ECP assumes any value in a partition is as good as another, BVA targets the edges of these partitions. Experience shows that programmers frequently make mistakes (like using < instead of <=) at the boundaries of input ranges.

The Core Principle

Instead of picking random values from an equivalence class, test cases are designed to test the minimum, the maximum, and the values just inside and outside these boundaries. If a system handles the extreme edge cases correctly, it is highly likely to handle the values in between correctly.

Values Tested in BVA:

  • Min - 1: Just below the lower boundary (Invalid)
  • Min: The exact lower boundary (Valid)
  • Min + 1: Just above the lower boundary (Valid)
  • Max - 1: Just below the upper boundary (Valid)
  • Max: The exact upper boundary (Valid)
  • Max + 1: Just above the upper boundary (Invalid)

Why Boundaries?

Off-by-one errors are incredibly common in programming. A loop might iterate one time too many, or an if (age > 18) might be written when the requirement was actually "18 and over" (if (age >= 18)). BVA explicitly hunts for these specific logical errors.

2. Step-by-Step Practical Example

Example: Age Constraint (18 to 60)

BVA Target

Scenario: Continuing from our ECP example, a user's age must be between 18 and 60 inclusive.

Boundary Condition Test Value Partition Type Expected Outcome
Min - 1 17 Invalid Error / Rejected
Min 18 Valid Accepted
Min + 1 19 Valid Accepted
... (values 20 through 58 are covered by ECP representative values) ...
Max - 1 59 Valid Accepted
Max 60 Valid Accepted
Max + 1 61 Invalid Error / Rejected

3. ECP vs. BVA: The Differentiation Table

While Equivalence Class Partitioning and Boundary Value Analysis are often used together, they have distinct purposes and methodologies in test case design.

Feature Equivalence Class Partitioning (ECP) Boundary Value Analysis (BVA)
Focus Area Divides the entire input domain into broad groups or classes. Focuses strictly on the exact edges or boundaries of those defined classes.
Test Selection Selects one representative value from anywhere inside the partition (middle, edge, etc.). Selects specific extreme values (min, min-1, max, max+1).
Primary Goal To reduce the total number of test cases while maintaining broad coverage. To find logical edge-case defects (like off-by-one errors) that developers frequently make.
Relationship Serves as the foundational step. You must define partitions before you can find their boundaries. Considered an extension or refinement of ECP. BVA depends on ECP being completed first.