Showing posts with label TEST CASES. Show all posts

SOFTWARE TEST CASE DESIGN

The design of tests for software and other engineered products can be as challenging as the initial design of the product itself. Yet, for reasons that we have already discussed, software engineers often treat testing as an afterthought, developing test cases that may “feel right” but have little assurance of being complete. Recalling the objectives of testing, we must design tests that have the highest likelihood of finding the most errors with a minimum amount of time and effort.

A rich variety of test case design methods have evolved for software. These methods provide the developer with a systematic approach to testing. More important, methods provide a mechanism that can help to ensure the completeness of tests and provide the highest likelihood for uncovering errors in software.

Any engineered product (and most other things) can be tested in one of two ways:

(1) Knowing the specified function that a product has been designed to perform, tests can be conducted that demonstrate each function is fully operational while at the same time searching for errors in each function;

(2) knowing the internal workings of a product, tests can be conducted to ensure that “all gears mesh,” that is, internal operations are performed according to specifications and all internal components have been adequately exercised. The first test approach is called black-box testing and the second, white-box testing.

When computer software is considered, black-box testing alludes to tests that are conducted at the software interface. Although they are designed to uncover errors, black-box tests are used to demonstrate that software functions are operational, that input is properly accepted and output is correctly produced, and that the integrity of external information (e.g., a database) is maintained. A black-box test examines some fundamental aspect of a system with little regard for the internal logical structure of the software.

White-box testing of software is predicated on close examination of procedural detail. Logical paths through the software are tested by providing test cases that exercise specific sets of conditions and/or loops. The “status of the program” may be examined at various points to determine if the expected or asserted status corresponds to the actual status.

At first glance it would seem that very thorough white-box testing would lead to “100 percent correct programs.” All we need do is define all logical paths, develop test cases to exercise them, and evaluate results, that is, generate test cases to exercise program logic exhaustively. Unfortunately, exhaustive

testing presents certain logistical problems. For even small programs, the number of possible logical paths can be very large. For example, consider the 100 line program in the language C. After some basic data declaration, the program contains two nested loops that execute from 1 to 20 times each, depending on conditions specified at input. Inside the interior loop, four if-then-else constructs are required. There are approximately 10 possible paths that may be executed in this program!

To put this number in perspective, we assume that a magic test processor (“magic” because no such processor exists) has been developed for exhaustive testing. The processor can develop a test case, execute it, and evaluate the results in one millisecond. Working 24 hours a day, 365 days a year, the processor would work for 3170 years to test the program. This would, undeniably, cause havoc in most development schedules. Exhaustive testing is impossible for large software systems.

White-box testing should not, however, be dismissed as impractical. A limited number of important logical paths can be selected and exercised. Important data structures can be probed for validity. The attributes of both black- and white-boxing can be combined to provide an approach that validates the software interface and selectively ensures that the internal workings of the software are correct.

21.2

ERROR CHECK LIST FOR INSPECTIONS

WALK THROUGHS IN TESTING

TESTING FOR SPECIALIZED ENVIRONMENTS PART ONE

TESTING FOR SPECIALIZED ENVIRONMENTS PART TWO

VALIDATION TESTING

SYSTEM TESTING

DEBUGGING AND TESTING

DEFECT AMPLIFICATION AND REMOVAL

ITERATIVE SPIRAL MODEL

STANDARD WATER MODEL

CONFIGURATION MANAGEMENT

CONTROLLED TESTING ENVIRONMENT

RISK ANALYSIS PART ONE

RISK ANALYSIS PART TWO

BACK GROUND ISSUES

SOFTWARE REVIEWS PART ONE

SOFTWARE REVIEWS PART TWO

SOFTWARE RELIABILITY

SAFETY ASPECTS

MISTAKE PROOFING

SCRIPT ENVIRONMENT

V MODEL IN TESTING

Business Rules of software testing part two

Data models are tools used in analysis to describe the data requirements and assumptions in the system from a top-down perspective. Traditionally, data models set the stage for the design of databases later on in the SDLC.

ER Diagram approach

An Entity-Relationship diagram shows entities and their relationships. The ER diagram relates to business data analysis and data base design.

There are three basic elements in ER models:

1. Entities are the "things" about which we seek information.

2. Attributes are the data we collect about the entities.

3. Relationships provide the structure needed to draw information from multiple entities.

DFD Diagram

Data flow diagrams can be used to provide a clear representation of any business function. They depict the movement and process steps of data and information. The DFD diagram is an ideal representation of a black box kind of scenario. A DFD diagram can be used for analysis of the system at a high level and can then be used to describe the system to the lowest level of detail or as required. Hence, you can have a clear top-down expansion of the system functionality.

A DFD uses the following to depict a business process diagram –

External Entity

An external entity is a source or destination of a data flow, which is outside the area of study. Entities, which receive or originate data, are represented in the DFD. Symbol is an oval, which contains a meaningful, unique identifier.

Process

A process represents a transformation or manipulation of data flows within a system.

UNIT TESTING PART ONE

UNIT TESTING PART TWO

UNIT TESTING PART THREE

GUI TESTING

WINDOWS COMPLIANCE GUI TESTING PART ONE

WINDOWS COMPLIANCE GUI TESTING PART TWO

WINDOWS COMPLIANCE GUI TESTING PART THREE

WINDOWS COMPLIANCE GUI TESTING PART FOUR VALIDATION TESTING

WINDOWS COMPLIANCE GUI TESTING PART FIVE CONDITION TESTING

WINDOWS COMPLIANCE GUI TESTING PART SIX GENERAL CONDITION TESTING

CONDITION TESTING

TESTING CONDITIONS PART ONE

TESTING CONDITIONS PART TWO

TESTING CONDITIONS PART THREE

TESTING CONDITIONS PART FOUR

SPECIFIC FIELD TESTING

USABILITY TESTING

INTEGRATION TESTING

INTEGRATION TESTING PART ONE

INTEGRATION TESTING PART TWO

INTEGRATION TESTING PART THREE

INTEGRATION TESTING PART FOUR

INTEGRATION TESTING PART FIVE

INTEGRATION TEST STANDARDS

INTEGRATION TEST STANDARDS PART TWO

Business Rules of software testing

Every system operates on business rules. If one were to document the business rules on which a system is based – it would be an exhaustive list to write down and track. Most business rules become implicit in the design and mind.

Business rules are typically implemented separately from presentation, application and data logic. To add to this, they are under constant change due to market requirements, user needs and software upgrades. Business Rules testing will help validate that your system is performing as expected, thus enforcing correct business practices on the user and the system code.

A typical way to gather business rules for your system is by talking to the business analyst and trying to understand what the analyst expects the system to do or how he/she expects the system to behave. A meeting with a developer will be helpful to understand how he/she thinks the system should behave. You may get conflicting ideas here, and it is important that these are sorted out before you have begun testing.

You can expand the table above to suit your needs in order to understand the various normal and alternative flows of the system. Again, mould the table and the data that you gather to suit the type of testing that you will do for your system. If you are doing unit testing, you will use more business rules like the first example (min char length), than like the third example (available only if Option A..)

Keep the following in mind:

1. Divide the system into modules as done by development.

2. For each module, take views based on the kind of testing you intend to do.

3. Direct your questions to the right people. E.g. If you are unit testing a product, a developer would be the right person to talk to. If you are system or user testing a product, a business analyst or end user would be able to answer your questions.

4. If you are short on time, document your business rules in order of decreasing priority.

5. Be specific with your questions. Run through the product or the system yourself, and make a list of questions, before you attempt to ask other team members.

6. Know the difference between a business rule and UI design. For e.g. a warning message to indicate that all information may be deleted is not a business rule, but a UI design.

7. Check if the business rules are incorporated in the system as code logic or are stored in the database and are made use of through a rules engine. This will help you organize your questions better.

WRITING SOFTWARE TEST CASES PART FOUR

Use Case Name: Login

Description: This use case explains the login functionality.

Actors: Administrator, User.

Main Flow: Login

Action

Response

1. Actor invokes the application.

1. Display login page with User ID, Password, Connect fields along with Login, Clear and Cancel buttons.

2. Actor enters User ID, Password and clicks on button.

Actor enters wrong User ID, see alternative flow 1.

Actor enters wrong Password, see alternative flow 2.

Actor chooses to connect to different database, see alternative flow 3.

Actor clicks on , see alternative flow 4.

Actor clicks on , see alternative flow 5.

2. Authenticate and display the home page.

Alternate Flow 1: Actor enters wrong User ID

Action	Response
1. Actor enters wrong User ID and password and clicks .	1. Display error message “Invalid User ID, Please try again”.

Alternate Flow 2: Actor enters wrong Password

Action	Response
1. Actor enters User ID and wrong password and clicks .	1. Display error message “Invalid Password, Please try again”.

Alternate Flow 3: Actor chooses different database

Action	Response
1. Actor enters User ID, Password and chooses to connect to a different database by selecting from the list.	1. Connect to the mentioned database and display home page.

Alternate Flow 4: Actor clicks on

Action	Response
1. Actor enters some information in the User ID, Password or Connect field and then clicks on	1. Clear the contents in the fields and position the cursor in the User ID field.

Alternate Flow 3: Actor clicks on

Action	Response
1. Actor clicks on button.	1. Close the login screen.

WRITING SOFTWARE TEST CASES PART THREE

Use Case Diagrams

As we have discussed earlier, a Use Case view explains the behavior of the system to end users, analysts and testers.

Designing a Use Case

With a small example, let us see how we can design a Use Case.

Suppose that we are designing an application which has the login screen functionality for authentication. The login screen is one of the most important in an application and the design for the same should be perfect.

The requirements for the login functionality are like this:

The login screen should have User Name, Password and Connect to fields.
The Connect to field is not mandatory. This field helps the user to connect to a particular database. If the user needs, he can use this option or by default it should connect to the database which we have pre-defined to be default.
Login, Clear and Cancel buttons should be available.
Depending on the login name, the user should be taken to the Administrator page or the normal user page.

We have our requirements clear. Now we need to approach to design use case for this.

As the definition goes, a use case is used to explain the behavior of the system. The UML Specification Document, UML Notation document or the UML Schematic document does not specify that a use case should be structured in a specific way. This leads to a debate of how the use case should be structured and designed. There are many formats and ways in which a use case can be designed and used. Here, I would be depicting the usually way in which I design a use case.

Use Case Section	Description
Name	An appropriate name for the Use Case.
Brief Description	A brief description of the use case’s role and purpose.
Actors	Persons who are users to the functionality.
Flow of Events	A textual representation of what the system does with regard to the use case. Usual notation for the flow of events are: Main Flow Alternative Flow Exceptional Flow
Special Requirements	A textual description that collects all requirements, such as non-functional requirements, on the use case, that are not considered in the use-case model, but that need to be taken care of during design or implementation.

Pre-Conditions	A textual description that defines any constraints on the system at the time the use case may start.
Post-Conditions	A textual description that defines any constraints on the system at the time the use case will terminate.

After having seen what the use case should typically contain, let us write the use case for the login requirements what we have defined.

WRITING SOFTWARE TEST CASES PART TWO

Theoretical Approach:

The theoretical approach will ensure that you have good comprehensive documents for the system at the end of the process.

What are Use Cases:

Use case is a description of the interaction between the user and the system. This can also be described as a way of using specific part of application’s functionality. A

complete collection of use cases describes all the ways that a user can manipulate a

software system.Use cases are a very powerful method of describing requirements.

They form basis for

Software Design
Test Cases
User documentation
GUI design

Use cases are written during the requirement phase of the software project. Use cases are written in a plain English language that user can easily understand. Business analysts that have been specially trained to develop use cases should write them. Rational Rose is a tool that can be effectively used to model and also write use cases. However most use cases are written in MSWord as everyone can easily access them.

Use Case Based Approach

Traditionally use cases have been used for requirement gathering. There has been a considerable shift to use them for testing, primarily because they model user role-play, movement, access rights and the interaction of one module to another – which is essentially what a test plan aims to do.

A use case defines the following –

1. Primary actor or actors

2. Goal

3. Scenarios used

4. Conditions under which a scenario occurs

5. Scenario result (goal delivery or failure)

6. Alternative scenarios or flows

The advantage of this approach is that you have two ready things in your hand

1. Documented system functionality

2. Evident module interaction

The development or creation of the use case will not be an easy task. Typical way to develop a use case is to pick up the system screen by screen or module wise and talk to the concerned developers or business analysts. Users can also be brought into the discussion at a later stage. Most users will give you valuable inputs to the type of data they input to the system. That in turn can give you an insight to the flows of the system.

Exemplary questions that you can ask users/developers are:

1. Inputs to a screen

2. User movements/warnings on a screen

3. Access rights to screen(s) or modules.

4. Access to screen(s) and exits from screen(s)

5. Database checks/updates.

6. Outputs from a screen

7. Alternative inputs or exits from the screen(s).

You can screen these questions or add some based on the testing you will be conducting for the system. For e.g. If you are carrying out GUI testing, you could do well if you add questions like length of the text box (this question could be answered well by a developer, rather than a user or a business analyst). If you are doing system testing, you could probably cut out the question on database check or updates. On the other hand, if you are doing system testing as part of a technology migration, you should enforce that database updates are verified.

WRITING SOFTWARE TEST CASES

Test cases have traditionally been used to test any system – software or otherwise. The test case may transform into a checklist, a comprehensive step by step guideline on information displayed by the system, or a simple black box scenario. In any case, a test case is a pure representation of what the system should and shouldn’t do.

A functional specification document on the other hand is documented the way the user would want the system to be or how he perceives the system to be. A typical functional system is dominated by user understandable language, screen shots and behavior. Business rules may also be documented as part of the document.

One can argue that it may be easier to test from a functional/requirements specification document than creating a separate test case document – since the basis of a test case document is the functional specification document.

However there are several known pitfalls with this approach. Some of them are -

Test scenario build up is difficult
Incomplete documentation of flows
Unit level approach rather than system level approach.
Business rules may not be apparent, though they are documented.

Typically a functional specification is the basis of developing test cases with user flows, business rules and special conditions thrown in.

Test cases without requirements specifications:

On the fly development, pressurized deadlines, changing user requirements, legacy systems, large systems, varied requirements and many more are known influences for many a project manager to discard or not update the functional specification document. While a project manager/requirements manager may be aware of the system requirements, this can cause havoc for an independent test team.

Dealing with a system that needs to be tested without functional specifications requires good planning, smart understanding and good documentation - lack of which caused the problem in the first place.

Before I begin describing some ways in which you can tackle the problem – here are a few questions you could be asking yourself

1. What type of testing will be done on the system– unit / functional / performance / regression?

2. Is it a maintenance project or is it a new product?

3. From where have the developers achieved their understanding of the system?

4. Is there any kind of documentation other than functional specification?

5. Is there a business analyst or domain knowledge expert available?

6. Is an end user available?

7. Is there a priority on any part of the functionality of the system?

8. Are there known high-risk areas?

9. What is the level of expertise of the development team and the testing team?

10. Are there any domain experts within your test team?

11. Are there any known skill sets in your team?

12. Are you considering manual/automated testing?

TEST CASE DESIGN PART SIX

Comparison Testing

In some applications the reliability is critical.
Redundant hardware and software may be used.
For redundant software, use separate teams to develop independent versions of the software.
Test each version with same test data to ensure all provisional identical output.
Run all versions in parallel with a real-time comparison of results.
Even if will only run one version in final system, for some critical applications can develop independent versions and use comparison testing or back-to-back testing.
When outputs of versions differ, each is investigated to determine if there is a defect.
Method does not catch errors in the specification.

Static Program Analysis

This strategy helps in identifying errors without executing the program. Peer reviewers and programmers will use this strategy to uncover probable static errors.

Mathematical Program Verification

If the programming language semantics are formally defined, one can consider program to be a set of mathematical statements. We can attempt to develop a mathematical proof that the program is correct with respect to the specification. If the proof can be established, the program is verified and testing to check verification is not required.

There are a number of approaches to proving program correctness. We will only consider axiomatic approach.

Suppose that at points P(1), .. , P(n) assertions concerning the program variables and their relationships can be made.

The assertions are a(1), ..., a(n).

The assertion a(1) is about inputs to the program, and a(n) about outputs.

We can now attempt, for k between 1 and (n-1), to prove that the statements between

P(k) and P(k+1) transform the assertion a(k) to a(k+1).

Given that a(1) and a(n) are true, this sequence of proofs shows partial program correctness. If it can be shown that the program will terminate, the proof is complete.

Static Program Analyzers

Static analysis tools scan the source code to try to detect errors.
The code does not need to be executed.
Most useful for languages which do not have strong typing.

It can check:

Syntax.
Unreachable code
Unconditional branches into loops
Undeclared variables
Uninitialised variables.
Parameter type mismatches
Uncalled functions and procedures.
Variables used before initialization.
Non-usage of function results.

Possible array bound errors.

Misuse of pointers.

RELATED POST

INTEGRATION TESTING PART ONE

INTEGRATION TESTING PART TWO

INTEGRATION TESTING PART THREE

INTEGRATION TESTING PART FOUR

INTEGRATION TESTING PART FIVE

INTEGRATION TEST STANDARDS

INTEGRATION TEST STANDARDS PART TWO

QUALITY TESTING

QUALITY ASSURANCE

QUALITY ASSURANCE PART TWO

QUALITY ASSURANCE SQA

QUALITY OF DESIGN OF TEST CASE

QUALITY MANAGEMENT IN SOFTWARE TESTING

TOOLS FOR QUALITY MANAGEMENT

STATICAL QUALITY ASSURANCE

ISO APPROACH TO QUALITY TESTING

TEST CASE DESIGN PART FIVE

Boundary Value Analysis.

It is observed that boundary points for any inputs are not tested properly. This leads to many errors. Large number of errors tend to occur at boundaries of the input domain.

Boundary Value Analysis(BVA) leads to selection of test cases that exercise boundary values.

BVA complements equivalence partitioning i.e. select any element in an equivalence class, select those at the ''edge' of the class.

Examples:

For a range of values bounded by a and b, test (a-1), a, (a+1), (b-1), b, (b+1).
If input conditions specify a number of values n, test with (n-1), n and (n+1) input values.
Apply 1 and 2 to output conditions (e.g., generate table of minimum and maximum size).
If internal program data structures have boundaries (e.g., buffer size, table limits), use input data to exercise structures on boundaries.

BVA and Equivalence partitioning both helps in testing the programs and covers most of the conditions. This method does not test the combinations of input conditions.

Cause Effect Graphing Techniques.

Translation of natural language descriptions of procedures to software based algorithms is error prone.

Cause-effect graphing attempts to provide a concise representation of logical combinations and corresponding actions.

Causes (input conditions) and effects (actions) are listed for a module and an identifier is assigned to each.
A cause-effect graph developed.
Graph converted to a decision table.
Decision table rules are converted to test cases.

TEST CASE DESIGN PART FOUR

Loop Testing

Loops are fundamental to many algorithms. Loops can be categorized as, define loops as simple, concatenated, nested, and unstructured. Loops can be defined in many ways.

To test the loops, following guidelines may be followed:

Simple Loops of size n:
Skip loop entirely
Only one pass through the loop
Two passes through the loop
m passes through loop where m
(n-1), n, and (n+1) passes through the loop. This helps in testing the boundary of the loops.

- Nested Loops
- Start with inner loop. Set all other loops to minimum values.
Conduct simple loop testing on inner loop.
Work outwards and take the next nested loop.

Continue until all loops are tested.

Concatenated Loops
- If independent loops, use simple loop testing.
If dependent, treat as nested loops.
Unstructured loops
- Don't test - redesign. This is known as poor design.

Black Box Testing

Functional tests examine the observable behavior of software as evidenced by its outputs, without any reference to internal functions. This kind of tests is from the user point of view, which means as if the user is testing as in the normal business functions.

· Black box tests normally determine the quality of the software. It is an advantage to create the quality criteria from this point of view from the beginning.

· In black box testing, software is subjected to a full range of inputs and the outputs are verified for its correctness. Here, the structure of the program is immaterial.

· Black box testing technique can be applied once unit and integration testing is completed.

· It focuses on functional requirements.

· It is compliment to the white box testing.

The main objective of the black box testing is to find:

incorrect or missing functions
interface errors
errors in data structures or external database access
performance errors
initialization and termination errors.

Some of the techniques used for black box testing are discussed below:

Equivalence Partitioning

The main objective of this method is to partitioning the input so that an optimal input data is selected. Steps to be followed are:

1. Divide the input domain into classes of data for which test cases can be generated.

2. Attempting to uncover classes of errors, if any.

3. Identify the both right and wrong input data while partitioning the data.

4. Test the program for all types of data.

Based on equivalence classes for input conditions.

An equivalence class represents a set of valid or invalid states

An input condition is either a specific numeric value, range of values, a set of related values, or a boolean condition.

Equivalence classes can be defined by:

If an input condition specifies a range or a specific value, one valid and two invalid equivalence classes defined.
If an input condition specifies a boolean or a member of a set, one valid and one invalid equivalence classes defined.

Test cases for each input domain data item developed and executed.

This method uses less number of input data compare to exhaustive testing. However, the data for boundary values are not considered.

This method though reduces significantly the number of input data to be tested, it does not test the combinations of the input data.

TEST CASE DESIGN PART THREE

Control Structure testing.

In programs, conditions are very important and testing such conditions is more complex than other statements like assignment and declarative statements. Basic path testing is one example of control structure testing. There are many ways in which control structure can be tested.

Conditions Testing

Condition testing aims to exercise all logical conditions in a program module. Logical conditions may be complex or simple. Logical conditions may be nested with many relational operations.

Can define:

Relational expression: (E1 op E2), where E1 and E2 are arithmetic expressions. For example, (x+y) – (s/t), where x, y, s and t are variables.
Simple condition: Boolean variable or relational expression, possibly proceeded by a NOT operator.
Compound condition: composed of two or more simple conditions, Boolean operators and parentheses along with relational operators.
Boolean expression: Condition without relational expressions.

Normally errors in expressions can be due to due to one or all or the following:

Boolean operator error
Boolean variable error
Boolean parenthesis error
Relational operator error
Arithmetic expression error
Mismatch of types

Condition testing methods focus on testing each condition in the program of any type of conditions. There are many strategies to identify errors.

Some of the strategies proposed include:

· Branch testing: Every branch is executed at least once.

· Domain Testing: Uses three or four tests for every relational operator depending on the complexity of the statement.

· Branch and relational operator testing: Uses condition constraints. Based on the complexity of the relational operators, many branches will be executed.

Example 1: C1 = B1 & B2

where B1, B2 are boolean conditions..
Condition constraint of form (D1,D2) where D1 and D2 can be true (t) or false(f).
The branch and relational operator test requires the constraint set {(t,t),(f,t),(t,f)} to be covered by the execution of C1.

Coverage of the constraint set guarantees detection of relational operator errors.

Data Flow Testing

First, a proper data flow diagram like control flow(see basis path flow) is drawn. Then selects test paths according to the location of definitions and use of variables. Any variables that have been defined in any program behaves in the following way:

D: define the variable, normally defined in declarative section,

U: use the variables which is defined earlier, in the program.

K: kill the variable, which is another state of the variable at any time of the execution of the program.

Any variable that is part of the program will undergo any of the above state. However, the sequence of states is important. We can avoid following anomalies during the program execution:

· DU: Normal,

· UK, UU: Normal,

· DD: Suspicious

· DK: Probable bug

· KD: Normal

· KK: Probable bug

· KU: bug

· UD: Normal

For example,

DU: Normal means a variable is defined first and then used in the program which is normal behavior of the data flow in the program.

DK: Probable bug means a variable is defined and then killed before using in the program. This may be bug as why the variable is defined and killed with out using in the program.

Programming Portal

Top Tabs