Building web services middleware with predictable service execution

This paper presents a set of guidelines, algorithms and techniques that enable web services middleware to achieve predictable execution times. Existing web service middleware execute requests in a best-effort manner. While this allows them to achieve a higher throughput, it results in highly unpredictable execution times, rendering them unsuitable for applications that require predictability in execution. The guidelines, algorithms and techniques presented are generic in nature and can be used, to enhance existing SOAP engines and application servers, or when newly being built. The proposed algorithms schedules requests for execution explicitly based on their deadlines and select requests for execution based on laxity. This ensures a high variance in laxities of the requests selected, and enables requests to be scheduled together by phasing out execution. These techniques need to be supported by specialised development platforms and operating systems that enable increased control over the execution of threads and high precision operations. Real-life implementation of these techniques on a single server and a cluster hosting web services are presented as a case study and with the resultant predictability of execution, they achieve more than 90% of the deadlines, compared to less than 10%, without these enhancements.