Automated unit testing of agent systems

Agent technology has been increasingly used for building sophisticated applications. Agents are autonomous and complex, making their correctness difficult to be verified. Although there has been some work on testing some aspects of an agent system, there is a lack of research for comprehensively testing an agent system automatically from a low-level stage.<br>This thesis explores an approach for automatically testing agent systems. In the approach an automated testing framework has been developed to allow for completely automated unit testing of an agent system, from generation of test cases, through automated implementation of test harnesses, running of test cases and production of a detailed report. Following the principle of model based testing, the approach tests each unit in the agent system against its details as specified in the unit’s design descriptor, which is part of the AOSE (Agent Oriented Software Engineering) design model of the system. An algorithm has been developed to comprehensively generate test case inputs taking into account value ranges of, and relationships between input variables. Test cases are executed to detect potential faults in a unit under test, based on a fault model we have developed that describes possible problems in different kind of units. Although test cases are generated automatically, we also allow for adding manual test cases with particular values to verify certain situations.<br>The evaluation results have shown that the tool can reveal previously unidentified problems in a system under test. Some of these problems cause substantial but difficult to detect errors at runtime. Some of the faults detected are false positives, however most of these can be avoided or can come with more precise notifications with the improvements of our testing framework.<br>