From Device-based to Device-free: Human Behaviour Sensing under Interference in Indoor Environment

Human behavior sensing is the process of capturing, analyzing, and interpreting human actions, movements, and interactions through various sensing technologies. This field has gained significant importance across multiple domains for its potential to yield valuable insights into human activities. However, the accurate interpretation of human behavior is hindered by prevalent challenges known as interference. The interference problem in indoor human behaviour sensing applications arises from diverse sources, encompassing the inherent complexities in human movements, signals with similar frequency and spatial constraints. It introduces distortions in the sensor data, potentially reducing behavior recognition accuracy. Indoor interference can be categorized into three types: 1)motion interference, 2)signal interference, and 3)multipath interference. Each presents unique challenges that need to be addressed for robust human behavior sensing systems. This thesis endeavors to tackle the aforementioned challenges. Its objectives encompass elucidating the underlying causes of interference, proposing novel methodologies to mitigate its effects, and validating the efficacy of these approaches through empirical studies. The key contributions summarized as follows: First, addressing the challenge of motion interference, we introduce SwingLoc, a device-based localization system. Localization is an important application , particularly in indoor settings, where walking constitutes a common and distinctive whole-body movement. SwingLoc capitalizes on the natural arm swinging motion during walking and utilizes the Doppler effects generated by acoustic signals to determine the user’s location. Leveraging off-the-shelf speakers commonly found in public indoor areas, SwingLoc tracks the direction of the wearable device towards these speakers across consecutive gait cycles. By solving a nonlinear least squares problem, SwingLoc accurately calculates the user’s position. Our real-world tests involving 6 users at 2 distinct locations demonstrate SwingLoc’s effectiveness, achieving an overall localization error rate of 85% within a 2-meter range, even under challenging conditions where a maximum of three speakers are available in the environment. These experiments underscore SwingLoc’s robustness and efficacy, highlighting its potential to facilitate fine-grained location-based services and enhance human well-being. To tackle the second challenge, we design an innovative device-free system MeshID leveraging unique orthogonal signal interference for user authentication. MeshID significantly improves the sensing sensitivity on RF signal interference, enabling the extraction of subtle individual biometrics through velocity distribution profiling (VDP) features derived from less distinct finger motions, such as drawing digits in the air. We design an efficient one-shot model retraining framework, achieving high model robustness and performance in complex environments. Through comprehensive real-world experiments, our results demonstrate that MeshID achieves an identification accuracy of 95.17% on average across three distinct indoor environments. The results indicate that MeshID outperforms the state-of-the-arts in identification performance with less cost. Last, a novel RF-based system RF-Eye is proposed to capture complete target shape at once in indoor environments without the need for prior training. We design and implement Linear Frequency Modulated (LFM) baseband signal with one directional antenna, making it suitable for capturing target shape in multipath. By harnessing narrow pulse signal reflections and Doppler Frequency Shift, RF-Eye is capable of rendering a full comprehensive image of the target shape. Implemented on a Universal Software Radio Peripheral (USRP) device, RF-Eye demonstrates a remarkable 100% success rate in our experimental trials, reliably obtaining the full shape even for highly intricate target objects. In conclusion, addressing and mitigating interference in human behavior sensing are critical for unlocking the full potential of this transformative technology. As the field continues to evolve, addressing interference challenges not only fortifies the reliability of behavioral insights but also fosters to the ethical and responsible deployment of these technologies across diverse real-world applications.