Nicola Ferro

Scientific Evaluation of a DLMS: A Service for Evaluating Information Access Components

Lecture Notes in Computer Science, 2006

In this paper, we propose an architecture for a service able to manage, enrich, and support the i... more In this paper, we propose an architecture for a service able to manage, enrich, and support the interpretation of the scientific data produced during the evaluation of information access and extraction components of a Digital Library Management System (DLMS). Moreover, we describe a first prototype, which implements the proposed service.

How robust are multilingual information retrieval systems?

by Nicola Ferro and T. Mandl

Proceedings of the 2008 ACM symposium on Applied computing - SAC '08, 2008

The results of information retrieval evaluations are often difficult to apply to practical challe... more The results of information retrieval evaluations are often difficult to apply to practical challenges. Recent research interest in the robustness of information systems tries to facilitate the application of research results for practical environments. This paper analyzes a large amount of evaluation experiments from the Cross Language Evaluation Forum (CLEF). Robustness can be interpreted as stressing the importance of difficult

How to Compare Bilingual to Monolingual Cross-Language Information Retrieval

Lecture Notes in Computer Science, 2007

The study of cross-lingual Information Retrieval Systems (IRSs) and a deep analysis of system per... more The study of cross-lingual Information Retrieval Systems (IRSs) and a deep analysis of system performances should provide guidelines, hints, and directions to drive the design and development of the next generation MultiLingual Information Access (MLIA) systems. In addition, effective tools for interpreting and comparing the experimental results should be made easily available to the research community. To this end, we

A Statistical and Graphical Methodology for Comparing Bilingual to Monolingual Cross-Language Information Retrieval

The Information Retrieval Series, 2008

A new methodology for the evaluation of MultiLingual Information Access (MLIA) systems is propose... more

Adding Multilingual Information Access to the European Library

Lecture Notes in Computer Science, 2007

A feasibility study was conducted within the confines of the DELOS Network of Excellence with the... more A feasibility study was conducted within the confines of the DELOS Network of Excellence with the aim of investigating possible approaches to extend The European Library (TEL) with multilingual information access, i.e. the ability to use queries in one language to retrieve items in different languages. TEL uses a loose coupling of different search systems, and deals with very short information items. We address these two characteristics with two different approaches: the “isolated query translation” approach, and the “pseudo-translated expanded records” approach. The former approach has been studied together with its implications on the user interface, while the latter approach has been evaluated using a test collection of over 150,000 records from the TEL central index. We find that both approaches address the specific characteristics of TEL well, and that there is considerable potential for a combination of the two alternatives.

CULTURA Outcomes for Improving the User's Engagement with Cultural Heritage Collections

by Nicola Ferro, N. Orio, and Chiara Ponchia

Procedia Computer Science, 2014

This paper reports on the outcomes of the CULTURA project that aims to improve user engagement wi... more This paper reports on the outcomes of the CULTURA project that aims to improve user engagement with digital cultural heritage collections. The IPSA digital collection has been used as a case study for the development of novel personalised access tools for a variety of different user categories. The evaluation results collected on user engagement with the used cultural heritage collections show that the users become involved and engaged.

Download

Annotations and Digital Libraries: Designing Adequate Test-Beds

Lecture Notes in Computer Science, 2007

The increasing number of users and the diffusion of Digital Libraries (DLs) has increased the dem... more The increasing number of users and the diffusion of Digital Libraries (DLs) has increased the demand for newer and improved systems to give better assistance to the user during the search of resources in collections managed by Digital Library Systems (DLSs). In this perspective, the annotations made on documents offer an interesting possibility for improving both the user experience of the DLS and the retrieval performance of the system itself. However, while different approaches based on annotations have been proposed, they still lack a full experimental evaluation, mainly because an experimental collection with annotation is missing. Therefore, this paper addresses the problem of setting an adequate experimental test-bed for DL search algorithms which exploit annotations, and discusses a flexible strategy for creating test collections with annotated documents.

Download

An Approach for the Construction of an Experimental Test Collection to Evaluate Search Systems that Exploit Annotations

Lecture Notes in Computer Science, 2007

This study addresses the lack of an adequate test collection that can be used to evaluate search ... more This study addresses the lack of an adequate test collection that can be used to evaluate search systems that exploit annotations to increase the retrieval effectiveness of an information search tool. In particular, a new approach is proposed that enables the automatic creation of multiple test collections without human effort. This approach takes advantage of the human relevance assessments contained

Annotations as a Tool for Disclosing Hidden Relationships Between Illuminated Manuscripts

Lecture Notes in Computer Science, 2007

... The use of annotations as a research tool in the humanities has been reported also in [6]. Th... more

A System Architecture as a Support to a Flexible Annotation Service

Lecture Notes in Computer Science, 2005

... A set of annotations is automatically Page 3. A System Architecture as a Support to a Flexibl... more

Graph-based automatic suggestion of relationships among images of illuminated manuscripts

Proceedings of the 2006 ACM symposium on Applied computing - SAC '06, 2006

Scientific research on illuminated manuscripts includes the disclosure of relationships among ima... more Scientific research on illuminated manuscripts includes the disclosure of relationships among images belonging to different manuscripts. Relationships can be modeled as typed links, which induce an hypertext over the archive. In this paper we present a formal model for annotations, which is the basis to build methods for automatically processing existing relationships among link types and exploiting the properties of the graph which models the hypertext. The result of this processing is twofold: new relationships can be suggested to help users in their research work, and the existing ones can be semantically validated to check for inconsistencies.

Download

Annotating illuminated manuscripts

Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries - JCDL '05, 2005

... anno-tations as if they were typed links able to carry additional information or data ... The... more

Annotations: A Way to Interoperability in DL

Lecture Notes in Computer Science, 2008

ABSTRACT This paper discusses how annotations and interoperability relate together and affect eac... more ABSTRACT This paper discusses how annotations and interoperability relate together and affect each other in digital library settings. We analyse interoperability and annotations in the light of the evolution of the field of digital libraries and provide recommendations for successful interoperable annotations towards the European Digital Library.

DESIRE 2011

ACM SIGIR Forum, 2012

Download

CLEF 2012

by Nicola Ferro and Tiziana Catarci

ACM SIGIR Forum, 2012

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM ha... more Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references. ... T. Catarci, P. Forner, D. Hiemstra, A. Pe¿nas, and G. Santucci, editors. Information Access Evaluation. Multilinguality, Multimodality, and Visual Analytics. Proceedings of the Third International Conference of the CLEF Initiative (CLEF 2012). Lecture Notes in Computer Science (LNCS) 7488, Springer, Heidelberg, Germany, 2012. ... P. Forner, J. Karlgren, and C. Womser- ...

Scientific Data of an Evaluation Campaign: Do We Properly Deal with Them?

Lecture Notes in Computer Science, 2007

This paper examines the current way of keeping the data produced during the evaluation campaigns ... more This paper examines the current way of keeping the data produced during the evaluation campaigns and highlights some shortenings of it. As a consequence, we propose a new approach for improving the management evaluation campaigns’ data. In this approach, the data are considered as scientific data to be cured and enriched in order to give full support to longitudinal statistical

The Importance of Scientific Data Curation for Evaluation Campaigns

Lecture Notes in Computer Science, 2007

Abstract. Information Retrieval system evaluation campaigns produce valuable scientific data, whi... more

CLEF 2005: Ad Hoc Track Overview

by Nicola Ferro, Gjf Jones, and T. Mandl

Lecture Notes in Computer Science, 2006

We describe the objectives and organization of the CLEF 2008 ad hoc track and discuss the main ch... more We describe the objectives and organization of the CLEF 2008 ad hoc track and discuss the main characteristics of the tasks offered to test monolingual and cross-language textual document retrieval systems. The track was changed considerably this year with the introduction of new document collections consisting of library catalog records derived from The European Library, with a non-European target language, and with a task offering word sense disambiguated data for groups interested in the impact of natural language processing on the performance of information retrieval systems. The track was thus structured in three distinct streams denominated: TEL@CLEF, Persian@CLEF and Robust WSD. The results obtained for each task are presented and statistical analyses are given.

Download

CLEF 2009 Ad Hoc Track Overview: TEL and Persian Tasks

Lecture Notes in Computer Science, 2010

The 2009 Ad Hoc track was to a large extent a repetition of last year's track, with the same thre... more

Download

CLEF 2004: Ad Hoc Track Overview and Results Analysis

Lecture Notes in Computer Science, 2005

... than ever, dealing with issues such as retrieval on semi-structured data, interactive retriev... more

Uploads

Papers by Nicola Ferro

Log In