Enhancing Construction Workforce Safety through Formulating an Augmented Reality Digital Twin System

<p dir="ltr">The construction industry grapples with persistent safety challenges, underscored by high fatality rates spanning decades. The intricacies inherent in construction projects pose formidable obstacles, impeding the sector’s capacity to effectively combat occupational injuries and fatalities. A particular concern lies in the inadequate Situational Awareness (SA) among onsite workers, posing a risk of failure to promptly identify and prevent imminent hazards. In response, heightened attention is now directed towards cutting-edge technologies, notably Deep Learning (DL) and Augmented Reality (AR). These technologies hold transformative potential and can actualise the concept of Digital Twin (DT) in the construction safety field. By harnessing real-time data analysis from construction sites, they can proactively unveil potential hazards, while visualising the hazard information aids onsite construction workers in forming immediate SA. This technological focus seeks to empower workers, facilitating swift hazard recognition and response, ultimately enhancing their ability to avert injuries and fatalities. </p><p dir="ltr">The aim of this thesis is to investigate the development of a real-time visual warning system that seamlessly integrates DL and AR technologies and to explore the practical implications of such systems for construction workers. To achieve these goals, Chapter 2 presents a comprehensive review of existing literature, focusing on DT-based sensing and visualisation technologies in construction safety. This involves emphasising their current applications and identifying critical gaps in the knowledge body. Through an in-depth analysis of 89 relevant papers, the review revealed an important finding: the construction industry has yet to fully harness and integrate innovative technologies into practical applications. The primary challenges stem from the dynamic and complex nature of construction activities, coupled with the immaturity of information synchronisation and processing. Key research focuses have been identified, including tracking and visualising dynamic hazards in complex Three-Dimensional (3D) construction environment, modelling relationships among onsite workers, the construction environment, and safety rules, and understanding how warnings influence onsite worker behaviour. This literature review serves as the foundation for the following chapters, which propose a development framework to address these challenges and advance the integration of DL and AR technologies in construction safety. </p><p dir="ltr">To track dynamic hazards and visualise hazard information, an innovative development framework for real-time visual warning systems was proposed in Chapter 3. The framework integrates key technologies, including Building Information Modelling (BIM), object tracking algorithms, Simultaneous Localisation and Mapping (SLAM) algorithms, and game development technology, to create an innovative real-time context-aware visual warning system. Subsequently, a tailored prototype was developed based on this framework, specifically designed for an AR Head-Mounted Display (HMD), showcasing advanced features for effective hazard communication. It was rigorously tested under three quasi-onsite hazard avoidance scenarios, namely, explosions, falls from heights, and struck-by accidents, providing valuable insights into its practical effectiveness. The results demonstrated the system’s capability to achieve high-precision 3D alignment between the virtual and real worlds. Moreover, it efficiently visualised hazard information over a large spatial range with minimal latency, empowering workers to promptly identify, comprehend, and avoid potential hazards. </p><p dir="ltr">Apart from the development of a real-time visual warning system and its feasibility for application in the construction environment, the usability of the system is a crucial aspect. In Chapter 4, the relationships among onsite workers, the construction environment, and construction regulations were modelled, followed by an assessment of the usability of a refined real-time AR system. Drawing from cognitive ergonomics theories, the model serves to reveal the cognitive characteristics of onsite workers in three main working modes, namely, operating, moving, and information retrieval. Accordingly, appropriate interaction design schemes are proposed to match these modes, facilitating intuitive access to AR information. An AR system with integrated work guidance, information retrieval, and real-time warning functions was developed to experimentally validate the model and the proposed interaction design schemes, providing insights into the system’s usability and its impact on the performance of onsite workers in simulated rebar-tying tasks. The results indicated a high usability rating for the designed system User Interface (UI), showcasing its effectiveness in improving the performance of onsite workers. This chapter not only establishes the success of the proposed model and interaction design but also illuminates how field workers leverage AR holograms and environmental information in their decision-making processes. </p><p dir="ltr">Moreover, the investigation of the impact of real-time holographic visual warnings on onsite construction worker behaviour is presented in Chapter 5. An innovative eye-tracking-based SA quantification method was proposed with the aim of automatically recording and analysing the practical implications of AR HMD-based warnings on construction workers’ actions and decision-making processes. Three metrics, i.e. Time to First Fixation (TFF), Dwell Time, and Minimum Distance, are employed to quantify the duration of different behaviours in response to worker performance in forming SA of their surrounding dynamic environments. Simulation experiments on construction object handling and assembly tasks confirm the viability of the proposed quantification method. The findings not only offer insights into how the developed real-time AR warning systems from previous chapters influence worker behaviour and their SA but also provide practical considerations for optimising the implementation of the warnings in construction contexts. The knowledge derived from this investigation establishes a foundation for further advancements in AR technology integration, contributing significantly to ongoing efforts to enhance safety in the construction industry. </p><p dir="ltr">Overall, this investigation into the development and usability of a real-time visual warning system integrating DL and AR technologies for construction safety has yielded valuable insights. By addressing dynamic hazard tracking, hazard information visualisation, workforce cognitive mechanism modelling, and visual warning impact quantifying, this thesis contributes significantly to enhance construction workforce safety in the construction industry. The proposed development framework, interaction design schema, and SA quantification methods provide practical solutions for addressing challenges in construction safety. As the construction sector continues to evolve, the knowledge derived from this thesis serves as a stepping stone for future advancements in AR technology integration, fostering a safer working environment for construction workers.</p>