Sonic Efforts: a preliminary model of feature extraction and sonification to express walking movements

Background: Movement based interaction using auditory feedback has the potential to sensitize people to the habitual ways of moving and the expression of effort in everyday physical activities such as walking. Much existing work tends to emphasise corrective auditory feedback to walking and gait analysis, to enhance or correct performance in sporting or rehabilitation contexts. An alternative approach is to encourage playful and expressive ways of standing and walking to raise kinaesthetic awareness. “Still Moving” Françoise, J., et al. (2017), draws attention to the act of standing, by zooming in on muscular activity to amplify stillness through auditory feedback. Key to this work is recognition of the subtlety and variant qualities in movement to be explored with sensor extracted features mapped to sound synthesis strategies. Contribution: Building on the Sonic Efforts art installation in 2019, this work formalises the ‘Prefix/Suffix Extraction model’ to combine the sensitivity to first person, variant and expressive qualities in movement (Sheets-Johnstone, M. 2011), with gesture to sound-action chunks (Godøy, R. I. 2009, 2011), to establish phases as salient to the expression of the footstep. By decomposing the footstep into a prefix-middle-suffix signal, we can control and explore various mappings of weight transfer through the articulation of the foot to sonic characteristics that may encourage the walker to play with their normal way of walking. This model led to this creative work, presented as a work-in-progress installation at TEI 2020. Significance: This work was peer reviewed and selected for TEI 2020 by the program committee. TEI is a key international conference for emerging approaches to Movement based and Tangible interaction. TEI is promoted via its website and supported by the Association for Computing Machinery (ACM). The ACM provides the computing field's premier Digital Library.