Evaluating knowledge management systems efficacy and effectiveness in a design science context

This paper describes the process of evaluating the efficacy and effectiveness of a knowledge management system (KMS) built as part of research for an Australian refrigeration manufacturing company, designed to enable the company to deal with identified strategic business problems, and proposes an evaluation framework for Design Science research. The artefact studied, here a KMS, is a key element in Design Science research. It is important to understand the evaluation process as this is central to determining the quality of Design Science research. Whilst there has been significant attention paid to the principles and enablement of Design Science research (Geerts, 2011; Hevner and Chatterjee, 2010a, b; Wang et al., 2011) there is significantly less attention given to attenuating evaluation (Venable, 2010), albeit that such evaluation is an integral part of Design Science research. This paper focuses on the process of evaluation of a specific KMS, but provides the context for more generalizability of the evaluation process with applications across other Design Science research.