SOFTWARE TESTING MATRICES PART TWO

Going through TESTING MATRICES PART ONE will give better understanding of this concept.

Cascading Test Matrices

The tests that occur on functional events vary based on the events themselves. If generic actions are used, it may be possible to include several functional events in the same matrix.

However, it is generally better to limit a matrix to a single functional event.

Including only one functional event on a matrix provides the following two advantages:

1• Tests can be customized for specific functional events.

2• Tests on the functional events can be the creation of new functional events which show a relationship between the events.

One functional event leading to the creation of another and leading to another will cause several matrices to be prepared. Properly prepared, they will demonstrate the cascading events illustrating how one event can create another event which can create yet another event.

An example of a cascading matrix is illustrated in the figure . This matrix is from an order entry billing system.

Click the picture for brooder view.


The first functional event is the order for a product placed by a customer. The type of actions that would occur on this is the initiation of the order, the acceptance of the order, the entry of the order, and the credit approval action.

That action creates a new functional event which is the formal approval of customer credit. The figure shows how the action in the first matrix cascades or points to the second matrix.

The tests for the functional event of approving credit involves such tasks as determining that the customer is an authorized customer, causing a decision about the customer’s credit.

If good action occurs, it creates a new functional event to ship a product. The figure shows how the action to ship a product creates a new functional event and a new matrix. This process would continue until the entire order entry billing application was complete.

In the creation of the test plan, IT people sometimes lose track of the interrelationship of functional events. The creation of the cascading test matrix reinforces the interrelationship of functional events and enables that aspect of systems to be better tested.

The coming post is regarding independent testing.

Life cycle testing can be understood in detail here.

The complete testing course can be browsed for further reference.



To Do Next: Thank you for visiting PROGRAMMING BLOG. If you liked the post, please subscribe to my blog via email or RSS FEED.You can contact me here for any specific feed back .

TESTING METRICES

The test matrix shows the interrelationship between functional events and tests. The completed test matrix defines the conditions that must be tested during the test process to verify the proper functioning of the application system. It does not represent the totality of testing because there may be types of tests that verify the structure of the system, such as verifying the correct use of program statements that are not included in the test matrix.

The left side of the matrix shows the functional events and the top identifies the tests that occur on those events. Within the matrix cells are the process that needs to be tested. During requirements, the process may be generalized and vague, but it must become more specific as the life cycle progresses.

The example illustrated in Figure is for the functional event of an employee getting a pay increase. The tests have been selected because each test:

1. Represents the actions that must be performed on this functional event in order for the employee to get a raise .
2. Represents a task that is performed individually.
3. Can be associated with the individual responsible to perform that action.
4. Is broad enough to limit the actions to a reasonable number .

Click the figure to enlarge.


In the figure example there are nine identified conditions that must be tested to determine whether or not employees’ pay increases are processed correctly by the application system.

Let’s examine the nine tests to illustrate the conditions that must be tested:

1 • Initiate Event Action

This action creates the functional event which will eventually give an employee a pay raise. Within the matrix is the process that needs to be evaluated. The process is for the supervisor to complete Form X. The information on the matrix must be condensed. If we were working with the application we would know, for example, that Form X was the organizational form for initiating a pay increase.

2 • Increase Approved Action

After the supervisor has completed the form it must be approved by management. The test condition indicates that management will initial Form X. Our test process or test condition would then be designed to verify that procedures are established so that this will happen. Note that this and the previous action are manual actions.

3 • Data Entry Action

The information on Form X is entered into the computer through a key entry process.The test condition is to verify that the keyed information is correct. Since this is in the early life cycle stages, we may not know all of the detailed information that will be keyed in or the verification process, but even in requirements we could substantiate that the requirements include a process to verify the accurate entry of data into the computer.

4 • Form Storage Action

Form X needs to be retained until it is no longer of value to the organization. The test condition states that the form should be stored for 90 days in the personnel department. Again, our test process would verify that this is reasonable and that the system process provides for this condition.

5 • Data Entry Validation Action

The verification that the information entered into the computer is correct. The test conditions outlined in the matrix state that the application system should validate that the pay increase amount is numeric, and that the increase is less than $100.

6 • Logical Validation Action

The test suggests that additional validation determination will be made beyond the data values. Three test conditions are outlined as: 1) a logical check that the employee exists, 2) the pay increase will be within the pay range the employee is authorized, and 3) that the pay increase is within plus or minus 15 percent of the amount the employee was earning prior to the increase.

7 • Updated Pay Record Action

The pay amount increase should be added to the employee’s pay rate so that the pay record will properly reflect the new amount. The test condition is to ensure that the changed pay rate amount in the storage area is performed correctly.

8 • Audit Trail Action

A record should be maintained on the increases provided the employee. The action shows that the increase in payroll should be maintained on a history file.

9 • Report Action
The results of computer processing should be sent back to the supervisor as a confirmation process. The test condition is to verify that the system provides for such a confirmation.

This testing matrix would be typical of one prepared during the requirements phase. The functional events, as well as the actions, will be used throughout the systems development life cycle.

However, the test conditions will become more specific as the life cycle progresses. For example, there is an action indicating that data will be entered into the computer. During requirements, how that will occur may not be known, so that the test condition must still be generalized as Figure Testing Matrix shows. However, as the system progresses through the life cycle, the testing matrix becomes more specific. Eventually, the testing matrix will be expanded to the point where test transactions can be prepared from the matrix information.


SOFTWARE TESTING FUNDAMENTALS

SOFTWARE TESTING FUNDAMENTALS PART TWO

SOFTWARE TESTING FUNDAMENTALS PART THREE

WHY SHALL WE TEST A SOFTWARE PRODUCT ?

TESTABILITY OF A SOFTWARE

APPROACHES TO SOFTWARE TESTING

TESTING STRATEGIES

ORGANIZING SOFTWARE TESTING

TESTING COMPLETION CRITERIA

SOFTWARE DEVELOPMENT PROCESS

SOFTWARE DEVELOPMENT LIFE CYCLE

PROJECT MANAGEMENT AND SOFTWARE TESTING

ECONOMICS OF SOFTWARE TESTING

DIFFERENT TEST LEVELS

TESTING TECHNIQUES

SPECIAL TEST TYPES

TESTING STANDARDS

ISO STANDARDS OF TESTING

TESTING PROCESS

TESTING PROCESS PART TWO

Scope of Software Testing

The scope of testing is the extensiveness of the test process. A narrow scope may be limited to determining whether or not the software specifications were correctly implemented. The scope broadens as more responsibilities are assigned to software testers.

Among the broader scope of software testing are these responsibilities:

1 . Software testing can compensate for the fact that the software development process does not identify the true needs of the user, and thus test to determine whether or not the user’s needs have been met regardless of the specifications.

2 . Finding defects early in the software development process when they can be corrected at significantly less cost than detected later in the software development process.

3 . Removing defects of all types prior to software going into a production state when it is significantly cheaper than during operation.

4 . Identifying weaknesses in the software development process so that those processes can be improved and thus mature the software development process. Mature processes produce software more effectively and efficiently.

In defining the scope of software testing each IT organization must answer the question, “Why are we testing?”

92.2

RELATED POST

SOFTWARE TESTING EFFECTING FACTORS

Organizing for software testing

The software developer is always responsible for testing the individual units (modules) of the program ensuring that each performs the function for which it was designed. In many cases, the developer also conducts integration testing

A testing step that leads to construction of the complete program structure. Only after the software architecture is complete, does an independent test group (ITG) become involved?

The role of an ITG is to remove the inherent problems associated with letting the builder test the thing that has been built. Independent testing removes the conflict of interest that may otherwise present. After all, personnel in the ITG team are paid to find errors.

How ever, the software developer does not turn the program over to ITG and walk away. The developer and the ITG work closely throughout a software project to ensure that thorough tests will be conducted. While testing is conducted, the developer must be available to correct errors that are uncovered.

The ITG is part of the software development project team in the sense that it becomes involved during the specification process and stays involved (planning and specifying test procedures) throughout a large project.

However, in many cases the ITG reports to the SQA organization, there by achieving a degree of independence that might not be possible if it were a part of the software development organization.

A Software Testing Strategy

The software engineering process may be viewed as a spiral.

Initially system engineering defines the roll of software and leads to software requirements and analysis, where the information domain, function, behavior, performance, constraints, and validation criteria for software are established. Moving inward along the spiral, we come to design and finally to coding.

To develop computer software, We spiral in along streamlines that decrease the level of Abstraction on each turn.

The strategy for software testing may also be viewed in the context of the spiral.

Unit testing begins at the vortex of the spiral and concentrates on each unit of the software as implemented in source code. Testing progresses by moving outward along the spiral to integration testing, where the focus is on design and the construction of the software architecture. Talking another turn outward on the spiral, we encounter

Validation testing where requirements established as part of software requirements analysis are validated against the software that has been constructed. Finally, We arrive at system testing where the software and other system elements are tested as a whole.

To test computer software, we spiral out along streamlines that broaden the scope of testing with each turn.Considering the process from a procedural point of view testing within the context of software engineering is a series of four steps that are implemented sequentially.

The steps are shown In Figure initially tests focus on each module individually, assuring that it functions as a unit hence the name unit testing. Unit testing makes heavy use of white-box testing techniques, exercising specific paths in a module's control structure to ensure complete coverage and maximum error detection.

Next, modules must be assembled or integrated to form the complete software package. Integration testing addresses the issues associated with the dual problems of verification and program construction. Black-box test case design techniques are most prevalent during integration, although a limited amount of white -box testing may be used to ensure coverage of major control paths. After the software has been integrated (constructed), sets of high-order test are conducted. Validation criteria (established during requirements analysis) must be tested.

Validation testing provides final assurance that software needs all functional, behavioral and performance requirements. Black-box testing techniques are used exclusively during validation.

The last high-order testing step falls outside the boundary of software engineering and into the broader context of computer system engineering. Software once validated must be combined with other system elements (e.g., hardware, people, and databases). System testing verifies the tall elements mesh properly and that overall system function/performance is achieved.

4
INTEGRATION TESTING PART ONE

INTEGRATION TESTING PART TWO

INTEGRATION TESTING PART THREE

INTEGRATION TESTING PART FOUR

INTEGRATION TESTING PART FIVE

INTEGRATION TEST STANDARDS

INTEGRATION TEST STANDARDS PART TWO

QUALITY TESTING

QUALITY ASSURANCE

QUALITY ASSURANCE PART TWO

QUALITY ASSURANCE SQA

QUALITY OF DESIGN OF TEST CASE

QUALITY MANAGEMENT IN SOFTWARE TESTING

TOOLS FOR QUALITY MANAGEMENT

STATICAL QUALITY ASSURANCE

ISO APPROACH TO QUALITY TESTING

Test Metrics in software testing

Metrics are the most important responsibility of the Test Team. Metrics allow for deeper understanding of the performance of the application and its behavior. The fine tuning of the application can be enhanced only with metrics. In a typical QA process, there are many metrics which provide information.

The following can be regarded as the fundamental metric:

1. Functional or Test Coverage Metrics.

2. Software Release Metrics.

3. Software Maturity Metrics.

4. Reliability Metrics.

5. Mean Time To First Failure (MTTFF).

6. Mean Time Between Failures (MTBF).

7. Mean Time To Repair (MTTR).

Functional or Test Coverage Metric. It can be used to measure test coverage prior to software delivery. It provides a measure of the percentage of the software tested at any point during testing.

It is calculated as follows:

Function Test Coverage = FE/FT

Where,

FE is the number of test requirements that are covered by test cases that were executed against the software

FT is the total number of test requirements

Software Release Metrics

The software is ready for release when:

1. It has been tested with a test suite that provides 100% functional coverage, 80% branch coverage, and 100% procedure coverage.

2. There are no level 1 or 2 severity defects.

3. The defect finding rate is less than 40 new defects per 1000 hours of testing

4. Stress testing, configuration testing, installation testing, Naïve user testing, usability testing, and sanity testing have been completed

Software Maturity Metric

Software Maturity Index is that which can be used to determine the readiness for Release of a software system. This index is especially useful for assessing release readiness when changes, additions, or deletions are made to existing software systems. It also provides an historical index of the impact of changes. It is calculated as follows:

SMI = Mt - ( Fa + Fc + Fd)/Mt

Where

SMI is the Software Maturity Index value

Mt is the number of software functions/modules in the current release

Fc is the number of functions/modules that contain changes from the previous release

Fa is the number of functions/modules that contain additions to the previous release

Fd is the number of functions/modules that are deleted from the previous release

Reliability Metrics

Reliability is calculated as follows:

Reliability = 1 - Number of errors (actual or predicted)/Total number of lines of executable code

This reliability value is calculated for the number of errors during a specified time interval.

Three other metrics can be calculated during extended testing or after the system is in production. They are:

MTTFF (Mean Time to First Failure)

MTTFF = The number of time intervals the system is operable until its first failure (functional failure only).

MTBF (Mean Time Between Failures)

MTBF = Sum of the time intervals the system is operable

MTTR (Mean Time To Repair)

MTTR = sum of the time intervals required to repair the system

The number of repairs during the time period .

RELATED POST

SOFTWARE QUALITY ASSURANCE AND CONTROL

SOFTWARE QUALITY AND COST ASPECT

STABLE PROCESS OF SOFTWARE TESTING

STABLE PROCESS OF SOFTWARE TESTING PART TWO

DEFECTS IN SOFTWARE TESTING

REDUCTION OF DEFECTS IN SOFTWARE TESTING

SOFTWARE TESTING AND EFFECTING FACTORS

SCOPE OF SOFTWARE TESTING

TESTING LIFE CYCLE PART ONE

TESTING LIFE CYCLE PART TWO

TESTING LIFE CYCLE PART THREE

SOFTWARE TESTING AND CONSTRAINTS WITH IN IT

TESTING CONSTRAINTS PART TWO

LIFE CYCLE TESTING

TEST METRICS

Independent Software Testing

Test Process

Testing verification and validation

Functional and structural testing

Static and dynamic testing

V model testing

Eleven steps of V model testing

Structural testing

Execution testing technique

Recovery Testing technique

Operation testing technique

Compliance software testing technique

Security testing technique