The software testing teams are rightfully referred to as quality assurance (QA) teams. Quality assurance of software can be achieved by testing its reliability, functionality, recoverability, resilience (security), interoperability and privacy.
Reliability implies that the software is functioning as it is expected by the business or customer. Since software is generally complex, the likelihood that all functionality and code paths will be tested is less and this can lead to the software being attacked.
Resiliency is the measure of how strong the software is to be able to withstand attacks, when an attacker is attempting to compromise it. Non-intentional and accidental user errors can cause downtime. Software attacks can also cause unavailability of the software. Software that is not highly resilient to attach will be susceptible to compromise such as injection threats, denial of service (DoS), data theft, memory corruption and when this occurs, the ability of the software to be able to recover its operations should alos be tested.
Recoverability is the ability for the software to restore itself to an operational state after downtime which can be caused accidentally or intentionally.
Interoperability testing validates the ability of the software to function in disparate environments.
Privacy testing is conducted to check that personally identifying information (PII), personal health information (PHI), personal financial information (PFI) and information that is exclusive to the owner of the information, is assured confidentiality and no intrusion.
1. TEST STRATEGY
The test strategy outlines the testing approach that will be undertaken. It is the main instrument that is used to inform and communicate testing issue with members of the software development team. Members are Project managers, testers, developers and management. It also includes information about the testing goals, methods, time needed, test environment configuration and needed resources. It describes the types of tests that are to be performed and the success/fail criteria at a high level. The strategy is high level in nature and is developed from conceptual design documents. In addition to functional design documents, it is highly advisable that in the formulation of a test strategy, the data classification, threat model, subject/object matrix, access control lists and considered, to assure security besides quality.
The following feature sets are carried out on this stage:
- Test levels
- Roles and responsibilities
- Environment requirements
- Testing tools
- Risks and mitigation
- Test schedule
- Regression test approach
- Test groups
- Test priorities
- Test status collections and reporting
- Test records maintenance
- Requirements traceability matrix
- Test summary
2. TEST PLAN
While the test strategy has in it the high level types of tests, the test plan is much more granular document that details the testing approach systematically. The test plan is more or less the workflow that a tester would perform. A test plan is used to ensure and verify that the software is reliable. For example meeting requirements, both functional and assurance (security) requirements. The three primary components of a test plan includes:
- Test requirements or responsibility
- Test methods
- Test coverage
Types of test plan
Master Test Plan:
A single high-level test plan for a project/product that unifies all other test plans.
Testing Level Specific Test Plans:
Plans for each level of testing.
- Unit Test Plan
- Integration Test Plan
- System Test Plan
- Acceptance Test Plan
Testing Type Specific Test Plans:
Plans for major types of testing like Performance Test Plan and Security Test Plan.
Test Plan Guidelines
- Make the plan concise. Avoid redundancy and superfluousness. If you think you do not need a section that has been mentioned in the template above, go ahead and delete that section in your test plan.
- Be specific. For example, when you specify an operating system as a property of a test environment, mention the OS Edition/Version as well, not just the OS Name.
- Make use of lists and tables wherever possible. Avoid lengthy paragraphs.
- Have the test plan reviewed a number of times prior to baselining it or sending it for approval. The quality of your test plan speaks volumes about the quality of the testing you or your team is going to perform.
- Update the plan as and when necessary. An out-dated and unused document stinks and is worse than not having the document in the first place.
3. TEST CASE
A test case takes the test requirements from the test plan and defined measurable conditions to validate that the requirements are indeed being met as expeced. Generally a test case contains a unique identifier, a reference to the requirements specifications that is being validated, any preconditions that need to be met, actions, test inputs and expected results. In addition to functional test cases, security test cases need to be defined as well.
Test case contains
- Test steps
- Expected result
- Actual result (Once tested)
Test Case best practices
When writing test cases, consider these things:
- Keep the title short.
- Include a strong description.
- Be clear and concise.
- Include the expected result.
4. TEST SCRIPT
While a test case answers the questions “What tests am I going to perform?”, a test script answers the questions “How am I going to perform the tests?” It is essentially the procedures that the tester will undertake to perform the test. Test scripts are developed using the test case, and for each test case, one or more test scripts need to developed. It is therefore imperative to ensure that security requirements are part of the test plan from which security test cases can be defined and these security test cases are then in turn.
5. TEST SUITE
Test cases don’t exist in silos but in groups and a collection of test cases makes up a test suite. It is usually organized logically by section, such as functional tests, performance tests. It is important to ensure that if such grouping exists, then the section for attesting the security of the software is not missed or overlooked.
6. TEST HARNESS
All the components that are necessary to conduct software testing are collectively referred to as a test harness. This includes the testing tools, test data samples, testing configurations, test cases and test scripts. Alternatively a test harness can be used as a test stub to simulate functionality and services that are still in development or not available in the test environment. In this respect, test harnessed are an important component of simulation testing. Test harnessed promote the principle of leveraging existing components as it can be reused by multiple projects, once it is set up.
A test harness may provide some of the following benefits:
- Increased productivity due to automation of the testing process.
- Increased probability that regression testing will occur.
- Increased quality of software components and application.
- Repeatability of subsequent test runs.
- Offline testing
- Access to conditions and/or use cases that are otherwise difficult to simulate (Example: load).