Software Testing [CTF] [07/05/2018]

Software Testing [CTF] [07/05/2018]

1. What is testing?

   • An exercise in Verification and Validation (V&V)
   • Ensuring a system runs as designed
   • Creating test cases to determine how a system performs
   • Attempting to replicate a known issue or uncover new bugs
   • Throughout development, ensure recently written modular code is compatible with the rest of the program
   • How you expect a system to act versus how the system does act

2. The Fundamental Test Process

   • 1. Planning and Control
     • Determining scope, risks, approach and objectives of testing
     • Implementing test policy/test strategy and deciding on exit criteria
     • Test control concerned with measuring results of reviews, testing as well as monitoring progress and making decisions
   • 2. Analysis and Design
     • Review the test basis (information we need to start the test analysis and create test cases (requirements, design specification, etc)
     • Identify test conditions, design the tests, evaluate testability of requirements and system, design test environment set-up
   • 3. Implementation and Execution
     • Develops and prioritizes test cases and creates test data for those tests - these instructions for tests are test procedures
     • With execution, test suites and test cases are excuted following test procedures, then re-execute the tests that previously failed to confirm a fix - known as confirmation testing
     • The test log is used to log the outcome of the test execution and record the identities and versions of the software under tests, then compare actual results and expected results
   • 4. Evaluating exit criteria and Reporting, exit criteria is based on risk assessment for the project and are taken into consideration when: Maximum test cases are executed with certain pass percentage, Bug rate falls below certain level, When deadlines are reached.
   • 5. Test closure activities
     • Done when software is delivered or other reasons such as project cancellation, all information gathered
     • To check all planned deliverables are delivered, incident reports have been resolved, handover test-ware to maintenance organization, evaluate how testing went and learn lessons for future releases and projects

3. The 7 Testing Principles

   • 1. Exhaustive (infinite) testing is not possible, optimal amount of testing is done based on the risk assessment of the application
   • 2. Defect clustering, states that a small number of modules contain most of the defects detected, thus, the Pareto principle: Approximately 80% of the problems are found in 20% of the modules. If the same tests are repeated over and over again, eventually the same test cases will no longer find new bugs
   • 3. Pesticide Paradox, repetitive use of the same pesticide mix to eradicate insects will over time lead to insects developing resistance to the pesticide, thereby becoming ineffective, same principle applies to software testing. Test cases should be regularly reviewed and revised, adding new and different test cases will help find new defects
   • 4. Testing shows the presence of defects, software testing reduces the possibility of undiscovered defects remaining in the software but even if no defects are found it is not a proof of correctness
   • 5. Absence of errors - fallacy, it is possible for software which is 99% bug-free to be unusable, can be the case if the software is tested thoroughly for the wrong requirement, as testing is also a check to see if it addresses the business needs. Finding and fixing defects does not help if the system build is unusable and does not fulfil the user’s needs and requirements
   • 6. Early testing, testing should start as early as possible in the SDLC so that any defects in the requirements or design stage are captured in early stages, and is much cheaper to fix a defect in the early stages of testing.
   • 7. Testing is context dependent, the way you test an e-commerce site will be different from the way you test a commercial off the shelf application, all developed software’s are not identical and different methodologies, approaches, techniques and types of testing may be made depending on application type.

4. Manifesto for Agile software development

   • Individuals and interactions over processes and tools
   • Working software over extensive documentation
   • Customer collaboration over contract negotiation
   • Responding to change over following a plan