The software testing teams are rightfully referred to as quality assurance (QA) teams. Quality assurance of software can be achieved by testing its reliability, functionality, recoverability, resilience (security), interoperability and privacy. Reliability implies that the software is functioning as it is expected by the business or customer. Since software is generally complex, the likelihood that all functionality and code paths will be tested is less and this can lead to the software being attacked. Resiliency is the measure of how strong the software is to be able to withstand attacks, when an attacker is attempting to compromise it. Non-intentional and accidental user errors can cause downtime. Software attacks can also cause unavailability of the software. Software that is not highly resilient to attach will be susceptible to compromise such as injection threats, denial of service (DoS), data theft, memory corruption and when this occurs, the ability of the software to be able to recover its operations should alos be tested. Recoverability is the ability for the software to restore itself to an operational state after downtime which can be caused accidentally or intentionally. Interoperability testing validates the ability of the software to function in disparate environments. Privacy testing is conducted to check that personally identifying information (PII), personal health information (PHI), personal financial information (PFI) and information that is exclusive to the owner of the information, is assured confidentiality and no intrusion. TESTING ARTIFACTS 1. TEST STRATEGY The test strategy outlines the testing approach that will be undertaken. It is the main instrument that is used to inform and communicate testing issue with members of the software development team. Members are Project managers, testers, developers and management. It also includes information about the testing goals, methods, time needed, test environment configuration and needed resources. It describes the types of tests that are to be performed and the success\/fail criteria at a high level. The strategy is high level in nature and is developed from conceptual design documents. In addition to functional design documents, it is highly advisable that in the formulation of a test strategy, the data classification, threat model, subject\/object matrix, access control lists and considered, to assure security besides quality. The following feature sets are carried out on this stage: \tTest levels \tRoles and responsibilities \tEnvironment requirements \tTesting tools \tRisks and mitigation \tTest schedule \tRegression test approach \tTest groups \tTest priorities \tTest status collections and reporting \tTest records maintenance \tRequirements traceability matrix \tTest summary 2. TEST PLAN While the test strategy has in it the high level types of tests, the test plan is much more granular document that details the testing approach systematically. The test plan is more or less the workflow that a tester would perform. A test plan is used to ensure and verify that the software is reliable. For example meeting requirements, both functional and assurance (security) requirements. The three primary components of a test plan includes: \tTest requirements or responsibility \tTest methods \tTest coverage Types of test plan Master Test Plan: A single high-level test plan for a project\/product that unifies all other test plans. Testing Level Specific Test Plans: Plans for each level of testing. \tUnit Test Plan \tIntegration Test Plan \tSystem Test Plan \tAcceptance Test Plan Testing Type Specific Test Plans: Plans for major types of testing like Performance Test Plan and Security Test Plan. Test Plan Guidelines \tMake the plan concise. Avoid redundancy and superfluousness. If you think you do not need a section that has been mentioned in the template above, go ahead and delete that section in your test plan. \tBe specific. For example, when you specify an operating system as a property of a test environment, mention the OS Edition\/Version as well, not just the OS Name. \tMake use of lists and tables wherever possible. Avoid lengthy paragraphs. \tHave the test plan reviewed a number of times prior to baselining it or sending it for approval. The quality of your test plan speaks volumes about the quality of the testing you or your team is going to perform. \tUpdate the plan as and when necessary. An out-dated and unused document stinks and is worse than not having the document in the first place. 3. TEST CASE A test case takes the test requirements from the test plan and defined measurable conditions to validate that the requirements are indeed being met as expeced. Generally a test case contains a unique identifier, a reference to the requirements specifications that is being validated, any preconditions that need to be met, actions, test inputs and expected results. In addition to functional test cases, security test cases need to be defined as well. Test case contains \tTitle \tDescription \tTest steps \tExpected result \tActual result (Once tested) Test Case best practices When writing test cases, consider these things: \tKeep the title short. \tInclude a strong description. \tBe clear and concise. \tInclude the expected result. 4. TEST SCRIPT While a test case answers the questions "What tests am I going to perform?", a test script answers the questions "How am I going to perform the tests?" It is essentially the procedures that the tester will undertake to perform the test. Test scripts are developed using the test case, and for each test case, one or more test scripts need to developed. It is therefore imperative to ensure that security requirements are part of the test plan from which security test cases can be defined and these security test cases are then in turn. 5. TEST SUITE Test cases don't exist in silos but in groups and a collection of test cases makes up a test suite. It is usually organized logically by section, such as functional tests, performance tests. It is important to ensure that if such grouping exists, then the section for attesting the security of the software is not missed or overlooked. 6. TEST HARNESS All the components that are necessary to conduct software testing are collectively referred to as a test harness. This includes the testing tools, test data samples, testing configurations, test cases and test scripts. Alternatively a test harness can be used as a test stub to simulate functionality and services that are still in development or not available in the test environment. In this respect, test harnessed are an important component of simulation testing. Test harnessed promote the principle of leveraging existing components as it can be reused by multiple projects, once it is set up. A test harness may provide some of the following benefits: \tIncreased productivity due to automation of the testing process. \tIncreased probability that regression testing will occur. \tIncreased quality of software components and application. \tRepeatability of subsequent test runs. \tOffline testing \tAccess to conditions and\/or use cases that are otherwise difficult to simulate (Example: load).