I am full professor in the department of Informatics of the University of Fribourg and director of the Human-IST institute. I am also heading the "human-building interaction" group in the smart living lab and the Swiss representative at the IFIP TC13.

After completing a PhD at the Swiss Federal Institute of Technology (EPFL), a postDoc in the USER group in IBM Almaden research center, one year of teaching and research in the university of Avignon, and a great experience as a usability engineer in a swiss start-up, I joined the University of Fribourg (more details on my background.

Other present and past projects: "Humanitics: Visual Analytics for All" (SNF Project); IM2 NCCR on Interactive Multimodal Information Management, and the Human Centered Design and Evaluation" individual project (IP2 phase 3), as well as "Integrated Multimodal Processing" individual project (IP1 phase 3); FP7 VisMaster CA, simpleUI, IM2.HMI, MeModules, NetSecurity, Risk Manager Dashboard, TotalRecall, Document Integration Project (IM2.DI), TIVIPOL, Database management and meeting analysis, SMAC, COMIND, Cobrowse.

The EmotiBoard (2011-2014) is an interdisciplinary project, leaded by Denis Lalanne and Fabien Ringeval at the department of Informatics and in collaboration with Prof. Juergen Sauer and Andreas Sonderegger from the department of psychology. The goal of this project is to develop a methodology for emotionally enriching remote collaborative interactions; members of virtual teams are indeed less productive and less affectively committed compared to collocated teams, in particular with regard to difficult work situation such as intercultural teamwork. The project involves both real-time emotion recognition from multimodal inputs (speech and electrodermal activity - EDA) and adapted emotional visual representation as an emotion feedback; one see the emotion of his/her remote partner. Various studies were performed with EmotiBoard to investigate its benefits for improving quality of remote collaborative interactions with an emotional feedback, in particular for emotion awareness.

The main goal of the Living in Smart Environments (LSE) project, financed by the Hasler Foundation, is to study subtle hand and arm gesture recognition for Human Computer Interaction in everyday use, i.e. systems able to recognize natural and economic gestures that take into account the painfulness and long-term efforts associated with gestures. I am currently supervising two PhDs financed on this project are investigating Tangible Gestures Interaction and the creation of a framework to coordinate and compare research works on gesture recognition..

RECOLA is a multimodal corpus of spontaneous collaborative and affective interactions in French Participants were recorded in dyads during a video conference while completing a task requiring collaboration. Different multimodal data, i.e., audio, video, ECG and EDA, were recorded continuously and synchronously. In total, 46 participants took part in the test, for which the first 5 minutes of interaction were kept to ease annotation. In addition to these recordings, 6 annotators measured emotion continuously on two dimensions: arousal and valence, as well as social behavior labels on five dimensions. The corpus allowed us to take self-report measures of users during task completion.

The overall goal of the Humanitics project, financed by the Swiss National Science Foundation, is to develop visual analytics tools for public organizations to enhance collaborative knowledge discovery, information exchange and communication. The project concentrated on the visualization of migration data over time. JFlowMap is the research prototype developed at the University of Fribourg to explore and evaluate spatio temporal flow map visualization techniques. Screenshots and videos can be found on here.

Weena is a personal information management (PIM) system enabling faceted navigation in a personal collection. Re-finding information items can be achieved in particular through the re- lated people (social facet) and temporal period (tem- poral facets) in addition to traditional hierarchical browsing and text search. Participants in the eval- uation effectively used those facets and expressed an interest for the approach. Faceted navigation is there- fore a viable and promising alternative to hierarchical browsing and text search, the two more traditional re-finding means in PIM.

Flowstrates is a new interactive visualization approach in which the origins and the destinations of the flows are displayed in two separate maps, and the changes over time of the flow magnitudes are represented in a separate heatmap view in the middle. This allows the users to perform spatial visual queries, focusing on different regions of interest for the origins and destinations, and to analyze the changes over time provided with the means of flow ordering, filtering and aggregation in the heatmap. In this paper, we discuss the challenges associated with the visualization of temporal origin-destination data, introduce our solution, and present several usage scenarios showing how the tool we have developed supports them.

The Portable Gestural Interface PyGmI is a smart tool to interact with a system via simple hand gestures. The user wears some color markers on his fingers and a webcam on his chest. The implemented prototype permits to visualize and navigate into presentation files, thanks to a tiny projector fixed on the user’s belt. The gesture recognition uses color segmentation, tracking and the Gesture and Activity Recognition Toolkit (GART). This article presents PyGmI, its setup, the designed gestures, the recognition modules, an application using it and finally an evaluation.

HephaisTK is a toolkit for rapid prototyping of multimodal interfaces. HephaisTK uses SMUIML, a simple markup language for describing human machine multimodal interaction scenarios and integration mechanisms. HephaisTK toolkit is released as open source software on SourceForge.

Tangible interfaces can be useful to establish links between our memory and information. We call them "MEMODULES". They roost abstract information in the real world through tangible shortcuts. MEMODULES are tiny tagged physical objects containing a link towards information sources that can be further accessed by several devices. We consider them as tangible hypermarks, i.e. embedded hyperlinks to abstract information attached to everyday physical objects. MEMODULES not only materialize information in the real world through tangible reminders, they also help controlling daily life devices. This project aims at designing, implementing and evaluating MEMODULES facilitating (a) information access and categorization and also (b) the control of everyday devices.

The TableMind supports collaborative mind mapping. It takes advantages of physical and digital worlds for collaborative tasks. The four members of the group sit around the table. The graphical user interface is covering a quadratic surface consisting of several different zones, each type of zone interpreting user input differently. Every user has a virtual Post-it block in the right-hand corner in front of him where he can create, write and draw an unlimited number of Post-its. He can then use the small handle to either drag these Post-its directly onto the mind map area in the center of the surface or stock (currently) up to five Post-its in the private storage area in front of him. The mind map area in the middle of the table is the common working space where the users create and discuss their mind map. This circular zone is rotatable. Post-its can be linked by overlapping them for a short fraction of time. Linked Post-its can then be freely moved within the mind map area whereas the linkage is updated automatically.

FaericWorld is a multimedia browser, which intends to visualize a large collection of heterogeneous multimedia documents. The major issues of FaericWorld are: 1) the cross-media navigation; 2) the validation of annotations on media and of links between documents; 3) the abstraction and creation of new documents, by hyper-linking existing ones. The archives imported, analyzed, and indexed by FaericWorld browser are composed of media (static documents, images, videos, or audio recordings), multimedia documents (slideshows or websites), people (person name, its publications, etc.) and events (conferences, meetings, etc.)

The SpiralView is a visualization tool for helping system administrators to assess network policies. The tool is meant to be a complementary support to the routine activity of network monitoring, enabling a retrospective view on the alarms generated during and extended period of time. The tool permits to reason about how alarms distribute over time and how they correlate with network resources (e.g., users, IPs, applications, etc.), supporting the analysts in understanding how the network evolves and thus in devising new security policies for the future. The spiral visualization plots alarms in time, and, coupled with interactive bar charts and a users/applications graph view, is used to present network data and perform queries. The user is able to segment the data in meaningful subsets, zoom on specific related information, and inspect for relationships between alarms, users, and applications.

JFriDoc is a document-centric meeting browser, based on the JFerret framework, that has been used for the Browser Evaluation Test (BET) in the context of IM2 and AMI projects. JFriDoc does not only consist of an adaptation of his ancestor, but it also explores novel concepts. In fact, JFriDoc abandon cross-meeting navigation in favor of extra-functionalities provided at the intra-meeting level, in order to handle larger meetings. Moreover, the meetings visualized in JFriDoc contain data and annotations that present various semantic links and overlaps: for example, an article might have been discussed several times and various participants could have been speaking at the same time. Managing more realistic data implies new visualizations for representing 1) multiple alignments between static document and speech; 2) relationships between discrete time and meeting; 3) overlapping of media at same time; and 4) multiple results to queries submitted by users.

FriDoc is a document-centric multimedia meeting browser.

TJASS, A SMART BOARD FOR AUGMENTING CARD GAME PLAYING AND LEARNING. The goal of Tjass is to extend card gaming with computational aids in a non intrusive and transparent way to support both beginner and expert players and to enrich their gaming experience. Tjass beginner’s assistance is directly available on the game board. In the traditional jass card game the players have to count and notate manually the scores. As well they need to observe continuously the status of the score to determine the winner team. In addition the rules of jass are complex and require keeping in mind the overview of all played and remaining cards. Therefore beginners often have difficulties to learn the game. Tjass will disburden the players of these tasks by (1) counting and displaying the score automatically in real time and (2) putting at beginner’s disposal a decision assistance to teach them the game.

The goal of SMAC project (Smart multimedia archives for Conferences), in collaboration with CERN and EIAFR, was to create a tool allowing the acquisition, storage, analysis and consultation of conference archives, from the acquisition of conference raw data (speaker audio/video, slideshow, articles, etc.); Data archiving and linking with existing digital libraries; to multimedia conference web browsing.

ELCANO, A TANGIBLE MULTIMEDIA BROWSER AND ORGANIZER- The goal of the Elcano project was to develop a simple tangible browser providing original views of the multimedia content of a storage medium, facilitating navigation using tangible sorting and filtering mechanisms and interactive visualizations. In this case, a memory stick is used as storage medium to provide mobility to the user.Elcano provides interactive visualizations to ease navigation through a great quantity of documents. To navigate in the views, physical widgets are used. Sorting and filtering algorithms as well as binding to personal objects are activated using physical tokens augmented with RFID tags.

PHONG, AUGMENTING VIRTUAL AND REAL GAMING EXPERIENCE- The goal of the Phong project, contraction for Physical Pong, is to implicate players physically in digital games. Tangible user interfaces have proved to be an intuitive and natural mean of interaction, particularly useful to reduce the gap between digital and physical worlds, since objects can co-exist in both dimensions. The purpose of Phong is to add both physical elements to the virtual world and virtual elements to the real world, making it a good example of mixed reality system.

X-Paint is an augmented table supporting tangible drawing activities. The interaction consisits of fast and easy to learn gesture that support two-handed control. The table allows the user to interact in a multimodal fashion, combining tangible interaction and voice recognition. The user can choose the colour and the size of the pen he/she wants to use by using multiple phidgets sliders and the rotation sensor. The user can choose the shape he/she want to draw by selecting the corresponding RFID-enabled object. Finally the user can use some vocal commands such as "begin draw", "select shape", "rotate shape" to perfom different drawing actions. This prototype was developed in the "multimodal interfaces" class I teach: much more prototypes can be found on: http://diuf.unifr.ch/diva/courses/multimodal/".

Inquisitor is an interface using heretix API that has been developed in order to correct document analysis results, in particular of Xed, on the basis of previous experiences. Inquisitor not only supports annotations correction, but it allows also to create manually new annotations on static documents further useful for creating links with other media in Faericworld.

IM2.DI Document-centric meeting recordings: We have recorded and annotated more than 20 meetings of about 15 minutes each. For each meeting a directory has been created on our media file server containing: The meeting global descriptor, audio/video files for each participants, and documents projected or discussed during the meetings. Annotations such as speech transcripts and documents content and structures, as well as alignments are also availiable. The meetings are a particular use-case: press reviews, i.e. meetings where participants discuss the cover page and the content of the newspapers of the day. Newspaper can be matched with speech transcript through citation, reference and thematic alignment. They contain small articles easy to segment. Thus, press reviews follow a structured agenda that should fit well document temporal alignment through document content alignment with speech transcripts. More information on http://diuf.unifr.ch/im2/.

Smart Meeting Rooms: Together with my colleagues, we developed 2 generations of smart meeting rooms to capture meetings with one microphone-camera pair per participants and other devices to capture documents and notes taken during meetings. A corpus of 22 meetings of press-reviews has been captured with the first generation of meeting room (distributed over several PCs with standards web cams). The most recent Meeting room, here in picture, uses acquisition cards on a single PC to capture 18 camera/microphone pairs.

CoBrowse is a computational assistant intended to collaborate with a user to provide WWW browsing and searching support. It does this by inferring a set of keywords that represent the user's current interests based on the content and sequence of pages visited. With the current interests as a guide, CoBrowse suggests what pages to visit next, displays maps of the WWW neighborhoods around the user's current page, and displays the user's history in terms of the user's previous tasks.

COMIND (Computer Aided Creativity and Multicriteria Optimization in Design) is the system I developped during my PhD thesis... to help designers to increase designers creative skills. COMIND helps designers search for consistent and optimal configurations of design parameters. This is built as a generic search and decision tool for any types of configuration tasks. Two fundamental processes are studied in this project: over-constrained problem where some preference and criteria must be taken out or relaxed in order for search to generate any solutions; and under-constrained problem where there are too many solutions thus additional criteria must be elicited from users to perform tradeoff analysis. After we introduced some algorithms to resolve conflicts and perform tradeoff analysis, we found that users are still not very motivated to make decisions about preferences and criteria. We then investigated visualization techniques to further enhance the decision context for users to more optimally determine the right criteria to relax, or to choose the solution using human criteria.