EdgeSOM: Distributed Hierarchical Edge-driven IoT Data Analytics Framework

With the rapid growth in the number of interacting IoT devices producing a huge amount of data, existing traditional systems are unable to handle the resulting data flow in such a way as to meet timeliness and performance requirements of critical services. Cloud computing systems have enabled us to access enormous computing power and storage capacity. However, despite its potential advantages, cloud computing is not always an ideal solution for real-time analytics services, where centralisation of computing resources has led to an increase in the separation between local devices and cloud partners, resulting in an increase in network latency, performance degradation and migration of the data away from its sources. To address these issues, a new paradigm is emerging, known as mobile edge computing (MEC), that enables the operation of highly demanding applications at the edge of the cellular network while meeting real-time response and low latency requirements. In this paper, we introduce EdgeSOM, a distributed and hierarchical MEC-based data analytics framework. EdgeSOM uses the combination of an enhanced Self-organising Map (SOM) and the Hierarchical Agglomerative Clustering (HAC) algorithm for distributed data clustering. EdgeSOM is fully distributed, such that MEC servers do not require a synchronisation server to cluster the data initially. The experimental evaluation shows that the EdgeSOM significantly reduces the network traffic of the aggregated IoT raw data to the cloud by up to 99.66% while achieving highly accurate analysis results.