Spring school of the 3eme cycle romand in Computer Science

Gesture Recognition
June 21st-22nd, 2010

Location: Room B130,
Perolles II (F7 on map) in the building 21,
University of Fribourg, Switzerland

In recent years, there has been a tremendous effort in research toward novel devices and techniques to improve the naturalness of interaction with computers. Gesture recognition is among the most promising attempt in this direction. Gestures are expressive, meaningful body motions involving physical movements of the fingers, hands, arms, etc., with the intent of conveying meaningful information or interacting with the environment. Numerous approaches have been applied to the problem of visual interpretation of gestures for HCI. Images of hands, geometric moments, contours, silhouettes, and 3D hand skeleton models are a few examples. In recent years, however, there has been an interest in incorporating the dynamic characteristics of gestures, which carry as many information as postures. The major goal of this school is to brush a state-of-the-art in gesture recognition, both at the recognition (detection, tracking, recognition), dialog and application levels. .


Program - Content of Tutorials - Location - Registration - Contacts

Program

Three internationally renowned speakers will present the state of the art in video based detecton and recognition of 3D gestures, including machine learning approaches. The school will also address techniques for recognizing 2D gestures (in particular pointing) on interactive surfaces, as well as the related HCI issues related to interaction with surfaces. Finally, the school will focus on applications of gesture recognition to musical applications.

  • June 21st
    • - 9:30 - 12:30 a.m. - Machine Learning Methods in Visual Recognition of Gesture and Action (R. Ronfard, INRIA) [presentation]
    • - 1:30 - 4:30 p.m. - Gesture Design and Implementation for Interacting on Surfaces (F. Berard, LIG) [presentation]
  • June 22nd
    • - 9:30 - 12:30 a.m. - Gesture controlled Musical Systems (F. Bevilacqua, IRCAM) [presentation]

Program - Location - Registration - Contacts

Content of the tutorials

Dr. Remi Ronfard, INRIA Grenoble

Machine Learning Methods in Visual Recognition of Gesture and Action

In this three-hour tutorial, we will review methods for learning models of human actions from visual examples, and recognizing them from video, using recent, state-of-the-art machine learning.

Gestures and actions are spatio-temporal patterns with internal structure and high complexity. The choice of suitable representations is often the most crucial aspect. In the spatial domain, actions and gestures can be represented with body models, with image models, or with bag of isolated features. In the temporal domain, they can be represented with templates, with grammars, or with bags of isolated features. By combining the spatial and temporal aspects of gesture and action, one is faced with a vast number of possible combinations.

The tutorial will survey the most useful and promising techniques for learning and recognition in each case, and discuss their advantages and limitations for real-life applications in face, gesture and full-body action recognition. In particular, we will examine how those different classes of methods can be adapted to achieve invariance with respect to viewing directions and performing styles.

Remi Ronfard

Rémi Ronfard, PhD, is a senior researcher in machine learning and computer vision at INRIA in Grenoble, France. After working for several years on multi-camera motion capture and action recognition in the MOVI team, he recently joined the LEAR team to investigate the topic of action recognition in movies.

In 2007-2008, he was the head of the Virtual Cinematography research team at Xtranormal Technology in Montreal, Quebec, where he developed automatic cinematography and movie editing tools for digital storytelling. Before that, he has worked in international research projects with IBM Research, Dassault Systèmes and INA as well as the MPEG expert group.


François Bérard, LIG Grenoble

Gesture Design and Implementation for Interacting on Surfaces

The advent of multi-touch interactive surfaces, such as the Apple iPhone, is breaking the status quo in Graphical User Interfaces. It is an opportunity to bring a more direct, gesture based, interaction from research experimentations to broad usage. But the design space for 2D gestures is very large and not all gestures are suitable to Human-computer interaction (HCI). This tutorial will be centered around the question "what makes a good gesture for 2D interactive surfaces?" Based on an overview of the reseach produced in the HCI litterature, and on our personnal experimentations, we will provide elements of answer to this question. We will acknowledge that gesture design should consider the technical limitations of the gesture tracking technology. The tutorial will thus include an overview of the tracking approaches that are available to the interaction designers.

François Bérard

François Bérard, PhD, is an associate professor at Grenoble Institute of Technology and a member of the Engineering of Human-Computer Interaction group of the LIG research lab (Laboratoire d'Informatique de Grenoble). He has been doing research on large interactive surfaces since his PhD and during scientific visits to the Xerox Palo Alto Reseach Center (PARC) and the MIT Media Laboratory. Since 2008, he splits his time between Grenoble and McGill University in Montreal to cooperate on research in the field of interaction in 3D.


Frédéric Bevilacqua, IRCAM Paris

Gesture controlled Musical Systems

We will review in this three-hour tutorial various gesture controlled musical systems. We will first present a brief state of the art of digital interfaces used for music expression. Then, we will focus on interactive systems integrating gesture recognition developed for musical applications, for interactive dance performances and interactive installations. Specific systems for the synchronization between gesture data and digital media (sound and/or video), based on machine learning techniques, will be explained. Finally, we will demonstrate some concrete examples of such interactive musical systems using various interfaces, prototypes but also tablet, webcam, game interfaces and mobile phone.

Frédéric Bevilacqua

Frédéric Bevilacqua is the leader of the Real Time Musical Interactions team at IRCAM - Institute for Music/AcousticResearch and Coordination in Paris (www.ircam.fr). He holds a master degree in physics (1991) and aPh.D. in Biomedical Optics (1998) from the Ecole Polytechnique Fédérale de Lausanne (Swiss Federal Institute of Technolgy in Lausanne). He also studied music at the Berklee College of Music in Boston (1992–1993) and has participated in various music and media arts projects. From 1999 to 2003 he conducted various research projects at the Beckman Laser Institute at the University of California Irvine. He joined IRCAM in October 2003 as researcher on gesture analysis for music and performing arts.



Program - Content of Tutorials - Location - Registration - Contacts


Location and schedule

University of Fribourg


Registration

Participation is free of charge for all members of the "Troisième Cycle" (i.e. EPFL, Universities of Fribourg, Geneva, Lausanne and Neuchâtel).

Access to the Registration Form


Organization and contacts
Denis Lalanne, University of Fribourg, Switzerland


This workshop is part of the 3eme cycle romand in Informatics
For more information, please refer to the conference website at:
http://diuf.unifr.ch/3e-cycle/ or http://www.cuso.ch/3e-cycle/informatique.html