department of informatics

FaericWorld

FaericWorld: A System for Managing and Exploring Multimedia Archives through Links and Static Documents

Keywords: Human machine interaction, Information visualization, Cross-media analysis and navigation, Meeting browsing

Summary:

  • FaericWorld is part of IM2.HMI project, which studies human-machine interfaces for visualizing, annotating and interacting with data recorded during meetings.
  • In particular, FaericWorld aims at designing and implementing a system that allows to
  1. navigate in a world composed of heterogeneous and multimedia documents
  2. visualize, validate and create document derived data (i.e. results of documents analysis, links and annotations)

The documents in this world consist of medias (static documents, video, audio and images), multimedia documents (e.g. slide shows), strictly related documents, i.e. clusters and events such as conference and meetings, and finally persons (descriptions and profiles). Search and navigation are based on those documents derived data, in order to retrieve and present relevant information to the user. The validation aims at accepting or correcting results of the various document analysis, annotations and links between related documents (e. g. document/speech thematic alignment). This validation task will allow 1) final users to navigate in a consistent world for retrieving information and 2) researchers to correct and refine algorithms of multimedia data analysis. Finally, the production of new documents, annotations and links will allow to associate existing data with new one.

  • The originality of this project consists in using links for browsing a multimedia archive of cross-connected data and in eliciting archive structure thanks to visualization and data clustering.

Period: The project has started on November 2003 and is supposed to end on June 2007

Fundings: The project is funded by IM2 project through IM2.HMI.

Participants:

Achievements:

  • FaericWorld [5]
    • database - currently, FaericWorld database contains already 22 meetings recorded at Fribourg, in Smart meeting Room. In the next few weeks, the database will be enriched with the entire MMM archive, which is composed of 171 fully annotated meetings, produced in the context of IM2 and AMI projects.
    • indexing and retrieval engine - FaericWorld is able to import and index meetings and to create links between annotations on multimedia documents. Nowadays, thematic alignment is the only automatic technique that is used to create links, but in the near future other methodologies as such as morphologic alignment will be added. Further, clustering will be integrated to improve data retrieval.
    • user interface - currently, FaericWorld interface uses a radial visualization to represent documents resulting by queries submitted to the systems (a default query is computed automatically by FaericWorld as entry point). The current result set could be expanded thanks to links, in order to discover other documents.
faeric world logofaeric world radviz
faeric world browsing
click to enlarge
  • JFriDoc [3]
    • JFriDoc is a multimedia browser based on JFerret package, developed at IDIAP institute, that is able to play meetings. It has been evaluated through the DIET, which aims at measuring the usefulness of static documents for browsing. JFriDoc is provided as meeting player in FaericWorld.
JFridoc document clickedJFridoc query highlighted
click to enlarge
  • Inquisitor [1, 5]
    • Inquisitor is an interface using heretix API that has been developed in order to correct document analysis results, in particular of Xed, on the basis of previous experiences [7, 8, 10, 12, 13]. Inquisitor not only supports annotations correction, but it allows also to create manually new annotations on static documents further useful for creating links with other media in Faericworld.
Inquisitor
click to enlarge
  • XED [2, 4, 6, 9, 11], see XED description
    • XED is a tool for re-structuring PDF documents and for further extracting high level abstractions such as the document logical structure. It allows to transform, purify and import PDF documents in FaericWorld archive.

Publications:

  1. F. Evéquoz , M. Rigamonti, D. Lalanne and R. Ingold, "Document inquisitor : un système de validation des structures et ’élicitation de modèles de documents", CIDE 2006, Fribourg, Switzerland, September 18 - 22 2006 , pp. 79-95
  2. J.-L. Bloechle, M. Rigamonti, D. Lalanne and R. Ingold, "XCDF : Un ormat canonique pour la représentation de documents", CIFED 2006, Fribourg, Switzerland, September 18 - 22 2006 , pp. 19-23
  3. M. Rigamonti, F. Evéquoz, D. Lalanne and R. Ingold, "Measuring the sefulness of static documents for multimedia meeting browsing", LMI 2006, Washington DC, USA, poster presentation.
  4. J.-L. Bloechle, M. Rigamonti, K. Hadjar, D. Lalanne and R. Ingold, "XCDF: A Canonical and Structured Document Format", DAS 2006, Nelson, New Zealand, pp. 141-152.
  5. M. Rigamonti, D. Lalanne, F. Evéquoz and R. Ingold, "Browsing multimedia archives through implicit and explicit cross-modal inks", MLMI 2005, Edinburgh, UK, pp. 114-125.
  6. M. Rigamonti, J.-L. Bloechle, K. Hadjar, D. Lalanne and R. Ingold, "Towards a Canonical and Structured Representation of PDF Documents through Reverse Engineering", ICDAR 2005, Seoul, Korea, pp. 1050-1054.
  7. E. Clavier, G. Masini, M. Delalandre, M. Rigamonti, K. Tombre and J. Gardes, "DocMining: A Cooperative Platform for Heterogeneous Document Interpretation According to User-Defined Scenarios", Graphics Recognition: Recent Advances and Perspectives, Revised Selected Papers, LNCS, Springer-Verlag Heidelberg, ISBN: 3-540-22478-5, GREC 2003, Barcelona, Spain, pp. 13-24.
  8. S. Adam, M. Rigamonti, E. Clavier, E. Trupin, J.-M. Ogier, K. Tombre and J. Gardes, "DocMining: A Document Analysis System Builder. 6th Internation Workshop", DAS 2004, Florence, Italy, pp. 472-483.
  9. M. Rigamonti, K. Hadjar, D. Lalanne and R. Ingold, "Xed: un outil pour l'extraction et l'analyse de documents PDF", CIFED 2004, La Rochelle, France, pp. 85-90.
  10. E. Clavier, S. Adam, P. Héroux, M. Rigamonti and J.-M. Ogier, "DocMining : Une plate-forme de conception de systèmes d'analyse de documents", CIFED 2004, La Rochelle, France, pp. 97-102.
  11. K. Hadjar, M. Rigamonti, D. Lalanne and R. Ingold, "Xed: a new tool for eXtracting hidden structures from Electronic Documents", DIAL 2004, Palo Alto, California, pp. 212-221.
  12. M. Rigamonti, O. Hitz and R. Ingold, "A Framework for Cooperative and Interactive Analisys of Technical Documents", GREC 2003, Barcelona, Spain, pp. 407-414.
  13. E. Clavier, G. Masini, M. Delalandre, M. Rigamonti, K. Tombre and J. Gardes, "DocMining: A Cooperative Platform for Heterogeneous Document Interpretation According to User-Defined Scenarios", GREC 2003, Barcelona, Spain, pp. 21-32.