Test collection for evaluating actionable knowledge graphs

Knowledge graphs (KG) can be used to enrich traditional search re- sults by inserting brief answers to directly respond to users' search needs. As the user needs on search engine diversify, the range of needs answered by KB should also be diversi ed. However, the resources for developing and evaluating KG generation technologies are still limited. In this paper we discuss the NTCIR-13 Actionable Knowledge Graph (AKG) task and its test collections. The task focuses on nding pos- sible actions related to input entities as well as the relevant properties of such actions. The NTCIR-13 AKG test collections include queries, entities, entity types, set of possible actions for entities, and relevant entity attributes. Finally, we discuss future directions for generating and evaluating actionable KGs.