Using Machine Learning to Identify Communal Worldwide Hand Gestures for Virtual and Hybrid Meetings Context

I am pleased to share that I will be supervising an Industrial Ph.D. student, Elizabete Munzlinger, who has received a grant of DKK 1.0 million from Innovation Fund Denmark. Elizabete will be working with the Office of CTO (OCTO) team of GN Audio A/S, a leading company in the collaboration business market, and the IT University of Copenhagen, a renowned institution for research and education in information technology.

Elizabete has been developing multimodal interaction methods to improve how users interact with computational systems. Her project will explore how machine learning can support a hand gesture vocabulary that promotes global standardization and inclusivity for new collaboration products and systems. She will use hand gesture recognition technology, which allows users to communicate and control devices using natural and intuitive hand movements, without touching anything. This technology can enhance user experience, safety, hygiene, and accessibility, especially for companies with international employees.

The project has three main objectives:

• To investigate how users from diverse backgrounds use hand gestures in virtual and hybrid meetings and which hand gestures they prefer for specific actions.
• To train a machine learning model to identify the most common and consistent hand gestures among cross-cultural users for controlling a given function in the interactive system or device.
• To propose a universal hand gesture dictionary that can support a global standardization for new collaboration products and systems that use this technology and foster understanding and well-being among users who work with international teams.

Hand gesture recognition technology has a huge potential in business meetings and collaboration products, as the demand for meetings and e-learning technology is growing worldwide due to the changes in work and study modes. Many industries are adopting this innovative resource, and some applications have already been launched, such as the Zoom platform and some collaborative business cameras. However, there is still room for improvement and innovation, as there is no shared standard vocabulary for hand gestures and some gestures may have different or offensive meanings in different cultures. Therefore, it is important to consider the cultural significance of gestures and create a conscious and communal vocabulary that can be universally understood and accepted.

To create hand gesture recognition products that can be used by global users from different backgrounds, it is not enough to ensure a high recognition rate. These products also need to provide a positive user experience, avoiding any embarrassment, misunderstanding, or offense that may discourage users from using the technology. Therefore, there is a need for a standardized hand gesture vocabulary that can achieve universal understanding, inclusivity, and acceptability. By conducting cross-cultural user studies, a hand gesture vocabulary can be carefully constructed to suit the needs and preferences of users from different cultures. This can increase the consumer confidence and the market potential of the products, as well as improve the state-of-the-art in hand gesture recognition for virtual and hybrid meeting contexts.

I am looking forward to working with Elizabete and our partners on this exciting and impactful project. I believe that this project will contribute to the advancement of human-computer interaction and the development of new products and services that can improve collaboration and communication among users from different backgrounds and cultures.