Neural mechanisms of dynamic face perception

Facial expressions are inherently dynamic, and the human visual system is sensitive to subtle changes in the temporal sequence of facial expressions. And yet, much of what we know about face perception is based on research that uses static images. Dynamic morphs, which transition between neutral and expressive photos, are a popular stimulus, as they offer increased experimental control in comparison to video recorded expressions. However, they fail to convey the complexity of facial motion. Deepfakes, photorealistic faces generated by artificial intelligence (AI), offer both experimental control and naturalistic motion. In a series of studies, the role of naturalistic facial motion in emotion perception was investigated by comparing these stimuli with the videos from which they were made. This thesis first investigates the “dynamic advantage”: the notion that emotion perception and associated neural responses differ for static compared to dynamic faces. Many studies exploring this phenomenon use dynamic morphs. Participants viewed dynamic morphs as less intense, and, depending on the emotion, less genuine, than video recorded and static faces. Electroencephalography (EEG) revealed that dynamic morphs triggered increased N400 amplitudes (indicating expectation violation), and decreased late positive potential (LPP) amplitudes (indicating reduced salience) compared to static and video recorded expressions. Together, these findings suggest that, paradoxically, dynamic morphs diminish the ecological validity of static faces by adding movement. Then, a novel deepfake database was created from a previously validated set of video-recorded emotional expressions. Unlike dynamic morphs, deepfakes were perceived as similarly intense and genuine compared to video recordings, and elicited largely similar neural responses. However, deepfakes elicited an increased N400 compared with video recordings. This suggests a neural sensitivity to subtle deviations in expectations from faces, which may not always reach awareness. Across both neuroimaging studies, frontal EEG activity in the delta frequency range (0.5-4 Hz) was increased for videos and deepfakes, but not static faces or dynamic morphs, implicating the involvement of frontal delta in the conscious perception of naturalistic facial motion. This thesis argues that dynamic morphs misrepresent facial dynamism, resulting in misleading insights about the neural and behavioural correlates of face perception. Despite some differences in neural responses, their similarities to videos make deepfakes a valuable asset for research where videos are unfeasible.