Researchers Seek Better Software Testing Techniques

Software systems in programmable items—from household appliances to cars and planes—must be reliable and high quality to avoid inconvenient or disastrous consequences. In order to improve software quality, computer science researchers at Harvey Mudd College and University of California, Santa Barbara (UCSB) are seeking to improve software quality assurance techniques and have received funding from the National Science Foundation (NSF).

Lucas Bang, assistant professor of computer science at Harvey Mudd, has teamed with Tevfik Bultan, professor and chair of computer science at UCSB, as co-principal investigator on the project “Automated Quantitative Assessment of Testing Difficulty.”

“There is no existing technique that can predict how challenging it will be to automatically test a piece of software,” says Bang whose research area is quantitative program analysis. “This project will address this open problem by providing an automated, scalable and quantitative approach for assessing software testing difficulty.”

Their work could lead to significant contributions to the field through the development of techniques for computing and composing two novel software complexity metrics, program path complexity and program path selectivity, to predict testing difficulty. The framework developed within the scope of the project will enable development of more effective software testing techniques based on better resource allocation for software quality assurance tasks.

“This will lead to improvements in software quality and reduction in software defects that cause dependability and security problems,” Bang says.

The research team will include graduate and undergraduate students—including those from underrepresented groups—who will be exposed to software quality assurance challenges and techniques. Research results will be incorporated in courses on both campuses, including in Bang’s courses on software verification, data structures and program development.

Since 2018, Bang has mentored 17 research students resulting in several papers with undergraduate lead authors either accepted, under submission or in preparation. He is also co-director, with computer science professor George Montañez, for an NSF-funded Research Experience for Undergraduates site at HMC focusing on computer systems.

Bang and Bultan will disseminate the knowledge, techniques and tools developed within the scope of the “Testing Difficulty” project through publishing in open literature and making available the software tools that are developed as open-source.