Smart personal sensing using on-device machine learning based on resource-constrained mobile devices

<p>With the rapid development of wearable and mobile devices such as wristbands, EEG headsets, smartwatches and smartphones, recent years have witnessed rapid increase in the demand of personal mobile sensing (PMS) applications. Through the various built-in sensors on these devices, PMS applications are able to exploit rich contexts from personal sensing data. Machine learning (ML) (i.e., classic ML or deep learning) plays a vital role in interpreting and understanding sensor data.</p> <p>As ML techniques usually require either manual feature engineering or heavy computation resources specially for training, most of existing solutions transmit sensor data from devices to clouds or edge servers to offload the workloads. However, since real-world PMS applications are intrinsically highly privacy-sensitive, user-specific (i.e., personal preferences or health conditions), low-latency interactive, and easily affected by local scenario changes (i.e., long-term behavior changes or ambient environment changes), the cloud-based approaches may suffer from severe data privacy concerns, compromised personalization, inferior model performance without continual training, and unacceptable network latency due to data transfer and model downloading.</p> <p>On-device ML (i.e., running ML with both training and inference on mobile devices) is proposed to effectively preserve sensing data privacy (i.e., no data transfer), and enable quick local model inference and update response for different on-device learning purposes (i.e., training from scratch, continual training or model adaptation), thus presenting a promising direction for real-world PMS applications. However, due to the challenges of highly sensing data dynamics, costly data collection, and resource-constrained commodity mobile devices in reality, existing on-device ML approaches may suffer from seriously inferior resource-accuracy efficiency for PMS applications.</p> <p>In this dissertation, we focus on the study of efficient on-device ML approaches for a broad range of smart PMS applications. To address the challenges associated with on-device ML, several novel approaches are proposed, aiming to achieve the state-of-the-art learning performance yet with much less resource cost. In particular, we propose a novel Gradient-based Movement Detection (GMD) unit that alleviates the effect of sensing data dynamics to improve learning performance in chapter 2; a novel ChainSGD-reduce approach that enables on-device collaborative learning to ensure learning efficiency and robustness in chapter 3; a novel Release-and-Inhibit Control (RIC) approach that transforms traditional Deep Neural Networks (DNNs) into resource-efficient model structures for efficient on-device ML in chapter 4.</p> <p>The main contribution of this thesis is to propose a complete on-device ML prototype system on commodity mobile devices. Leveraging a set of proposed novel approaches, the system can achieve on-device training and inference with superior resource-accuracy efficiency to bridge the gap between ML and privacy-preserving PMS applications, improving interactivity efficiency with individuals. It is envisioned that this research moves an important step towards the promising on-device learning paradigm. To further extent, the proposed system can be applied to build intelligent edge systems and Internet of Things (IoT) applications.</p>