Controlling viewpoint from markerless head tracking in an immersive ball game using a commodity depth based camera

We present the use of commodity depth based cameras to control viewpoint from passive unencumbered head tracking. This allowed participants to maintain correct perspective while playing an immersive computer ball game, without the need to wear devices, wires or markers. While viewpoint update is more generally useful than stereoscopy it has traditionally used tracking equipment of expense and complexity unsuitable for home use. The current explosion of depth based cameras into the home game market could provide a cheaper, simpler and more elegant solution. While updating viewpoint from a depth camera has been demonstrated it has not previously been studied in a multi-player or fast moving game or undergone accurate measures of latency, task performance or user experience. We integrated low cost commodity hardware including depth based cameras, computers, projectors and stereo glasses, computer games software, a laptop and large projection surfaces. System latency and user performance and experience were measured across a number of viewpoint conditions. System latency of the viewpoint and hand tracking were measured by analysing high speed video footage. The number of goals scored and saved gave a measure of task performance. Subjective impressions of experience were taken from questionnaire and post interview.