This book constitutes the thoroughly refereed post-workshop proceedings of the 8th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2009, held in Brisbane, Australia, in December 2009. The aim of the INEX 2009 workshop was to bring together researchers in the field of XML IR who participated in the INEX 2009 campaign. During the past year, participating organizations contributed to the building of large-scale XML test collections by creating topics, performing retrieval runs and providing relevance assessments. The workshop concluded the results of this effort, summarized and addressed issues encountered, and devised a work plan for the future evaluation of XML retrieval systems. The 42 full papers presented together with 3 invited papers were carefully reviewed and selected from 49 submissions. They have been divided into sections according to the eight tracks of the workshop, investigating various aspects of XML retrieval, from book search to entity
This book constitutes the thoroughly refereed post-workshop proceedings of the 9th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2009, held in Vught, The Netherlands, in December 2009. The aim of the INEX 2010 workshop was to bring together researchers in the field of XML IR who participated in the INEX 2010 campaign. During the past year, participating organizations contributed to the building of large-scale XML test collections by creating topics, performing retrieval runs and providing relevance assessments. The workshop concluded the results of this large-scale effort, summarized and addressed issues encountered, and devised a work plan for the future evaluation of XML retrieval systems. The 37 full papers presented were carefully reviewed and selected from 42 submissions; they are organized into sections corresponding to the nine tracks of the workshop, investigating various aspects of XML retrieval, from book search to entity.
This book constitutes the thoroughly refereed proceedings of the 7th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2008, held at Dagstuhl Castle, Germany, in December 2008. The aim of the INEX 2008 workshop was to bring together researchers who participated in the INEX 2008 campaign. Over the year leading up to the event, participating organizations contributed to the building of a large-scale XML test collection by creating topics, performing retrieval runs, and providing relevance assessments. The workshop concluded the results of this large-scale effort, summarized and addressed the issues encountered, and devised a work plan for the future evaluation of XML retrieval systems. The 49 papers included in this volume report the final results of INEX 2008. They have been divided into sections according to the seven tracks of the workshop, investigating various aspects of XML retrieval, from book search to entity ranking, including interaction aspects.
This book constitutes the thoroughly refereed post-proceedings of the 4th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2005, held at Dagstuhl Castle, Germany, in November 2005. The book presents 41 revised full papers, organized in topical sections on methodology, multiple retrieval, ad-hoc retrieval, relevance feedback, natural language queries, and more heterogeneous retrieval, interactive retrieval, document mining, and multimedia retrieval.
This book constitutes the thoroughly refereed post-workshop proceedings of the 10th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2011, held in Saarbrücken, Germany, in December 2011. The 33 revised full papers presented were carefully reviewed and selected for presentation at the workshop from 36 submissions. The papers are organized in 5 research tracks on book and social search, Xdata centric, question answering, relevance feedback, and snippet retrieval.
This open access book covers all facets of entity-oriented search—where “search” can be interpreted in the broadest sense of information access—from a unified point of view, and provides a coherent and comprehensive overview of the state of the art. It represents the first synthesis of research in this broad and rapidly developing area. Selected topics are discussed in-depth, the goal being to establish fundamental techniques and methods as a basis for future research and development. Additional topics are treated at a survey level only, containing numerous pointers to the relevant literature. A roadmap for future research, based on open issues and challenges identified along the way, rounds out the book. The book is divided into three main parts, sandwiched between introductory and concluding chapters. The first two chapters introduce readers to the basic concepts, provide an overview of entity-oriented search tasks, and present the various types and sources of data that will be used throughout the book. Part I deals with the core task of entity ranking: given a textual query, possibly enriched with additional elements or structural hints, return a ranked list of entities. This core task is examined in a number of different variants, using both structured and unstructured data collections, and numerous query formulations. In turn, Part II is devoted to the role of entities in bridging unstructured and structured data. Part III explores how entities can enable search engines to understand the concepts, meaning, and intent behind the query that the user enters into the search box, and how they can provide rich and focused responses (as opposed to merely a list of documents)—a process known as semantic search. The final chapter concludes the book by discussing the limitations of current approaches, and suggesting directions for future research. Researchers and graduate students are the primary target audience of this book. A general background in information retrieval is sufficient to follow the material, including an understanding of basic probability and statistics concepts as well as a basic knowledge of machine learning concepts and supervised learning algorithms.
This book provides a comprehensive discussion and new insights about linear optimization of content metrics to improve the automatic Evaluation of Text Summaries (ETS). The reader is first introduced to the background and fundamentals of the ETS. Afterward, state-of-the-art evaluation methods that require or do not require human references are described. Based on how linear optimization has improved other natural language processing tasks, we developed a new methodology based on genetic algorithms that optimize content metrics linearly. Under this optimization, we propose SECO-SEVA as an automatic evaluation metric available for research purposes. Finally, the text finishes with a consideration of directions in which automatic evaluation could be improved in the future. The information provided in this book is self-contained. Therefore, the reader does not require an exhaustive background in this area. Moreover, we consider this book the first one that deals with the ETS in depth.
This book constitutes the thoroughly refereed post-conference proceedings of the 6th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2007, held at Dagstuhl Castle, Germany, in December 2007. The 37 revised full papers presented were carefully reviewed and selected for presentation at the workshop from 50 initial submissions. The papers are organized in an ad hoc track and 6 topical sections on book search, XML-mining, entity ranking, interactive, link-the-wiki, and multimedia.
This open access book summarizes the first two decades of the NII Testbeds and Community for Information access Research (NTCIR). NTCIR is a series of evaluation forums run by a global team of researchers and hosted by the National Institute of Informatics (NII), Japan. The book is unique in that it discusses not just what was done at NTCIR, but also how it was done and the impact it has achieved. For example, in some chapters the reader sees the early seeds of what eventually grew to be the search engines that provide access to content on the World Wide Web, todays smartphones that can tailor what they show to the needs of their owners, and the smart speakers that enrich our lives at home and on the move. We also get glimpses into how new search engines can be built for mathematical formulae, or for the digital record of a lived human life. Key to the success of the NTCIR endeavor was early recognition that information access research is an empirical discipline and that evaluation therefore lay at the core of the enterprise. Evaluation is thus at the heart of each chapter in this book. They show, for example, how the recognition that some documents are more important than others has shaped thinking about evaluation design. The thirty-three contributors to this volume speak for the many hundreds of researchers from dozens of countries around the world who together shaped NTCIR as organizers and participants. This book is suitable for researchers, practitioners, and students--anyone who wants to learn about past and present evaluation efforts in information retrieval, information access, and natural language processing, as well as those who want to participate in an evaluation task or even to design and organize one.
