Principles of Software Testing

Principles of Software Testing

As software development techniques have advanced during the last decades, some basic principles of testing have also been established. Describing theoretical ideas and practical hints, these principles can be seen as a basic guideline for both, testing and coding.


Principle 1 - Testing shows presence of defects 

It does not verify that no more bugs exist, even if no more errors can be found. Testing is not a prove that the system is free of errors. But what if , you work extra hard , taking all precautions & make your software product 99% bug free .And the software does not meet the needs & requirements of the clients.

I.e. Software Testing reduces the probability of undiscovered defects remaining in the software but even if no defects are found, it is not a proof of correctness



Principle 2 - Exhaustive testing is impossible

If you were to test the entire possible combinations project EXECUTION TIME & COSTS will rise exponentially.

Hence, one of the testing principle states that EXHAUSTIVE testing is not possible. Instead we need optimal amount of testing based on the risk assessment of the application.

 
I.e : Microsoft Windows Calculator.

Suppose someone gives you work to test calculator functionality. And you decided to start with addition operation. You enter 1+0 = and get output 1.next you enter 1+1= and get output 2. How far will you check? Calculator ( Microsoft windows 7) accepts 16 digits. You have to check for all possibilities up to 1+ 9999999999999999 = .

After that, you can move to 2+0 =, 2+1=  and so on.
Like this, you will get to 9999999999999999+9999999999999999 =

Next, you should try for decimal values: 1.0+0.1= , 1.0+0.2 = and so on.
It is not enough yet, we still have to check by illegal inputs to ensure that, it will be handled properly. Editing operation is also should be tested like,

1 {backspace}2+0= and this will also apply for all above inputs.

Like this, there are so many inputs that we can’t achieve. Even you achieve above inputs, still multiplication, subscription and division etc. are left.

The aim of this example is only to give idea that it is impossible to test completely a software, even software as simple as a calculator. During this test, if you decided to skip any condition just because of time limitation or unnecessary to test, you have decided not to test the program completely.

Principle 3 - Early Testing

Testing activities should begin as early as possible within the software life cycle. They should be repeated regularly and have its’ own agenda. Early testing helps detecting errors at an early stage of the requirements, design or development process which simplifies error correction (and reduces the costs for this work)

When the defect find in the later stages, it will cost high to fix it. So it’s better to find out and fixed the defects early stages.

Principle 4 - Defect Clustering

Defect clustering means more defects concentrated or found within one or very few functional areas of the software.  There are various reason that can lead to Defect clustering, below are the list of key reasons.

- Defect clustering can occur because of poorly defined requirements for a particular functionality.
-  Complexity of the functionality.
- Developer's lack of experience.
- When different developers have fixed several defects on the same functionality.  Even defect fixes can introduce new defects, over a period of time the functionality can become defect prone.



Principle 5 - Pesticide Paradox

 If the same tests are repeated over and over again, eventually the same test cases will no longer find new bugs.   

The Reasons for this is

        -          The Practical Impossibility of testing all Possible Scenarios.

        -          The functionality of the application changes over time.

To overcome this, the test cases need to be regularly reviewed & revised, adding new & different test cases to help find more defects.  But even after all this sweat & hard work in testing, you can never claim you product is bug free.




Principle 6 - Testing is context dependent

 The same tests should not be applied across the board. Different software products have varying requirements, functions and purposes. Testing is done differently on different contexts.

 

I.e.: Flight control software need to be tested in a different way to a game; an on-line information retrieval facility will require less rigor than an e-commerce site.

 

Principle 7 - Absence of errors – Fallacy

 Just because no errors have been found does not mean that the software is ready for production. The software may conform to the specification, but the specification was not approved by the one person in the organization who uses the particular routines all day, every day. Before any tests have been run, there are no failures, but that does not mean that we are ready to enter into production.

i.e. Finding and fixing defects does not help if the system build is unusable and does not fulfill the user’s needs & requirements