Multi-Objective Optimization of Software Test Cases Using Evolutionary and Soft Computing Techniques

Abstract

Software testing occurs concurrently during the software development to identify errors as early as possible and to assure that changes made in software did not affect the system pessimistically. Since software testing is a time consuming, complex, full of uncertainty and expensive activity, delivering the human safety, medical, robotics software without proper testing may lead to potentially much higher cost than that of testing. However, during the development phase, the test suite is updated and tends to increase in size. Due to the resources and time constraints for re-executing large test suites, it is important to develop techniques to reduce the effort of regression testing. Some challenges for testing software systems are the effort, cost, complexity, ambiguity, test data adequacy involved in optimizing test cases. Optimization of the test cases before testing will surely cut down efforts, cost and improve the quality of software testing. Complexity, risks, cost, fuzziness and multi-criteria test cases fitness evaluation makes test case optimization an essential part of software testing. Several approaches have been proposed for reducing the effort, cost and improve the quality of regression testing such as test case selection, classification, filteration, prioritization, and test suite minimization. Test suite minimization techniques aim at identifying and eliminating redundant, obsolete, unfit, ambiguous test cases from the suite. Test suite minimization techniques identify a subset of test cases from suite, required to re-test the changes in the software. Test case selection is a selection of a subset of test cases from the test suite based on some test criteria. Test case prioritization techniques schedule test cases for execution in an order to increase the early fault detection, but the problem of identifying an optimized test suite with maximum coveragebility in cost efficient manner is the critical problem of software testing. There are several objectives of test case optimization like maximum number of defect detecting capability, minimum test design efforts/cost, minimum execution cost, maximum coveragebility of codes and client requirements, maximum mutant killing score and so forth. However, most of the test cases optimization approaches are single objective, but single objective formulation of test cases optimization problem is not sufficient and not meeting the objectives of testing. In multi-objective test case optimization, some objectives are conflicting in nature such as coveragebility of one objective will effect other objective while considering all objectives concurrently. Test cases optimization is NP-Complete, data and knowledge driven, human intensive, incomplete, vague, and multi-dimensional search space partitioning, and reduction problem. The approaches presented in the literature have various drawbacks and limitations such as resources, time, etc. In this work, multi-objective optimization of test cases with an emphasis on assessment of fitness and ambiguity of test cases on several parameters concurrently, by integrating evolutionary and soft computing approaches is presented. New three-tier framework has been proposed for multi-faceted test cases classification and selection using fuzzy synthesis, fuzzy entropy and ant colony optimization approach to overcome resource limitations. The experimental study has been done to identify an optimal solution of multi-faceted test cases classification and selection. In this empirical study, we have also measured the efficiency of all three steps and analyzed the size of regression test suite. For validation of proposed framework, simulation of proposed framework is done in MATLAB™ software. Data sets used for experimentation is taken from the Software Infrastructure Repository (SIR). The results of empirical study clearly show that the precision, recall, F1 score and accuracy of the algorithms increases as we move in next stage. The third stage of proposed framework has highest precision, recall, F1 score and accuracy values among all three stages for all the case studies. Results of experimental study shows that proposed conduit framework reduces the cost and efforts, uncertainty, and the number of test cases in test suite to be exercised. The results of empirical evaluation show that our three-tier framework efficiently finds out an optimal subset of test cases from large pool of test cases. The proposed framework enhances the quality of software testing by reducing the ambiguity, efforts, and size of the test suite.