Gestural interaction paradigms for smart spaces (GrIPSs)

Principal Investigators

Prof. Dr. Susanne Boll, University of Oldenburg (Homepage)
Prof. Dr.-Ing. Antonio Krüger, Saarland University (Homepage)

The aim of this project is to explore the gestural modality in pervasive computing spaces and establish the foundations for the design of consistent and scalable gesture-based interaction in such environments. By focusing on the design and assessment of gestures, our goal is to contribute to developing scalable interaction paradigms for pervasive environments. We also aim to identify the dimensions and aspects necessary to describe and characterize gestures within a pervasive computing context, allowing for their evaluation in different contexts. Two reference scenarios, namely Smart Homes and Smart Retail Shops, will be explored to demonstrate our findings’ scalability and ecological validity across various settings, including private and public real-life environments. Ultimately, the results of this project will inform the design and assessment of gestures that can be applied across individuals, spaces, and situations.

Main Research Question

Can we achieve cross-domain consistency based on gestures recommended in the literature?

Sub-Research Questions
  1. How do users make mid-air gestures?
  2. What is the inter-person and intra-person variability in gesture performance?
PhD Students

Masoumehsadat Hosseini

University of Oldenburg

Donald Degraen

Saarland University

Publications
Towards a Consensus Gesture Set: A Survey of Mid-Air Gestures in HCI for Maximized Agreement Across Domains

Hosseini, M., Ihmels, T., Chen, Z., Koelle, M., Müller, H. and Boll, S.
ACM CHI 2023
Evaluation of the transferability of gesture vocabulary between domains

Smart Devices as Proxy Objects for Virtual Reality Position Paper

A., Makhsadov, D., Degraen, A., Zenner, and F., Daiber
ACM CHI 2021
A framework that allows a smart device to be tracked in VR such that users can use its features for input, control, and feedback.



VRySmart: a Framework for Embedding Smart Devices in Virtual Reality

N., Nilsson, A., Zenner, A. L., Simeone, D., Degraen, and F., Daiber.
ACM CHI 2022
VRySmart: a Framework for Embedding Smart Devices in Virtual Reality

Everyday Proxy Objects for Virtual Reality.

F., Daiber, D., Degraen, A., Zenner, T., Döring, F., Steinicke, O., J., Nunez, and A. L., Simeone.
ACM CHI 2021
VRySmart: a Framework for Embedding Smart Devices in Virtual Reality


Haptic Proxies for Virtual Reality: Success Criteria and Taxonomy

A., Makhsadov, D., Degraen, A., Zenner, F., Kosmalla, K., Mushkina, and A., Krüger
ACM CHI 2021
Criteria for successful use of haptic proxies in virtual reality, present a taxonomy of techniques using haptic proxies.

  1. Masoumehsadat Hosseini, Tjado Ihmels, Ziqian Chen, Marion Koelle, Heiko Müller, and Susanne Boll. 2023. Towards a Consensus Gesture Set: A Survey of Mid-Air Gestures in HCI for Maximized Agreement Across Domains. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–24. https://doi.org/10.1145/3544548.3581420
  2. Akhmajon Makhsadov, Donald Degraen, André Zenner, Felix Kosmalla, Kamila Mushkina, and Antonio Krüger. 2022. VRySmart: a Framework for Embedding Smart Devices in Virtual Reality. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 358, 1–8. https://doi.org/10.1145/3491101.3519717
  3. Niels Christian Nilsson, André Zenner, Adalberto L. Simeone, Donald Degraen, and Florian Daiber. 2021. Haptic Proxies for Virtual Reality: Success Criteria and Taxonomy. In Workshop on Everyday Proxy Objects for Virtual Reality at CHI ’21, May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 5 pages.
  4. Akhmajon Makhsadov, Donald Degraen, André Zenner, and Florian Daiber. 2021. Smart Devices as Proxy Objects for Virtual Reality Position Paper. In Workshop on Everyday Proxy Objects for Virtual Reality at CHI ’21, May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 5 pages.
  5. Florian Daiber, Donald Degraen, André Zenner, Tanja Döring, Frank Steinicke, Oscar Javier Ariza Nunez, and Adalberto L. Simeone. 2021. Everyday Proxy Objects for Virtual Reality. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, Article 101, 1–6. https://doi.org/10.1145/3411763.3441343