Search behavior and relevance judgments in different search environments

<p>This research studies search behavior and relevance judgments in different search environments (SEs) on search engine result pages (SERPs) using information foraging theory (IFT) and search economic theory (SET). SEs were operationalized based on their differing types of display and methods of query input into desktop computers, tablet computers, and mobile smartphones. The purpose is to understand and explain how search behavior and relevance judgments differ with different SEs. Two user studies were conducted to 1) understand the influence of display type on search behavior using IFT, 2) understand the influence of display type on relevance judgments, and 3) understand the influence of query input method on search behavior using SET.</p> <p>The main findings are: 1. SERPs with more relevant search results led to deeper scroll depths and more user interactions on mobile and desktop displays. Search behavior varied across different display types. 2. Document judgments differed between professional assessors and users in the different SEs; the agreement between assessors and users was poor. In term of assessing documents, the increase in document relevance increased document reading time on mobile and tablet devices, in agreement with previous findings. 3. Users were categorized into different categories based on their typing speed. Faster typists entered more queries and spent less time typing. However, a relationship between typing speed and topic familiarity could not be established.</p> <p>Generally, empirical evidence showed that increased information patch qualities increased user interactions, and increased typing effort reduced the number of queries issued. The findings of the current user studies were limited by the number of participants (36-48) and the amount of time (< 60 minutes) allocated to each experiment. We suggest that future work considers SE as a factor for relevance judgments and evaluations of information retrieval systems and consider including typing behavior as part of the analyses.</p>