Diversifying microblog posts

Microblogs have become an important source of information, a medium for following and spreading trends, news and ideas all over the world. As a result, microblog search has emerged as a new option for covering user information needs, especially with respect to timely events, news or trends. However users are frequently overloaded by the high rate of produced microblogging posts, which often carry no new information with respect to other similar posts. In this paper we propose a method that helps users effectively harvest information from a microblogging stream, by filtering out redundant data and maximizing diversity among the displayed information. We introduce microblog posts-specific diversification criteria and apply them on heuristic diversification algorithms. We implement the above methods into a prototype system that works with data from Twitter. The experimental evaluation, demonstrates the effectiveness of applying our problem specific diversification criteria, as opposed to applying plain content diversity on microblog posts.