Methods to Study Interaction Paradigms in Pervasive Computing Environments:
Current study methods in applied psychology and human-computer interaction have been developed to guide and evaluate the design of individual devices. In the future, it will be necessary to study an increasing number of different devices but, more importantly, study virtually unlimited combinations of devices and contexts. Current methods, including usability testing, focus groups, and controlled experiments, do not scale with this increasing diversity. Promising directions include “research in the large,” simulating users’ behavior, and the development of models allowing transfer of findings from one context to another. Traditional lab studies and lab-based observation methods are mostly unsuited to investigate these settings, because they are hardly able to cover the various aspects and huge dynamics of the context. Future interactive systems in pervasive environments will lead us to evaluate novel paradigms “in the wild” – as this has been called recently – to increase the realism of these evaluations. Shadowing, the permanent observation of a study participant by a human experimenter, used the gold standard field observation technique in pervasive environments, although it lacks scalability and is quite obtrusive. More advanced, unsupervised user study techniques, such as logging or the Experience Sampling Method, were proposed and successfully applied. The sensor-based observation approach has shown its benefits and advantages in several structured investigations and field studies of various complexities and dynamics. In these techniques, the obtrusive human observer is replaced by sensors or in situ, self-reporting techniques. While these techniques are scalable and unobtrusive, they feature less situational details. Consequently, more research is needed in field studies, where the users, applications, and services are investigated in their natural environments. Studies done without any interference or visible presence from an experimenter could give us an incredibly realistic view of how technologies and interfaces are used in practice.
Research Questions
- With which methods do we successfully run unsupervised studies in the wild for larger interactive systems to get situated insights into the interaction in pervasive computing environments?
- What are the limitations of these observations regarding reliability of the observation and its interpretation for the success of an interaction paradigm?
- How can we employ Virtual Reality and Augmented Reality settings to evaluate interaction paradigms in the lab in which the immersive VR/AR simulates the “study in the wild”?
- How can we develop simulations for pervasive computing environments and within them the interaction paradigms to simulate human behavior based on psychological and physiological plausible models to evaluate their interaction behavior even before a prototype has been built?
- How do we reliably employ novel sensor technology to increase the explanatory power of unsupervised observation (vital signals, EEG, fNIRS, gaze detection) in comparison to observed user studies?
We expect individual research projects to investigate how accurate and comparable unsupervised sensor-based observations are, when compared, e.g., to a traditional shadowing observation approach. We expect experimental studies as well as data analytics on logging data to determine, classify and potentially also predict human behavior when interacting under specific interaction paradigms. Research projects may build and employ human models to simulate human interaction in pervasive computing environment to scale up evaluations. Some research may explore how situating people for experiments in environments with AR and VR technology allows to study interaction paradigms very close to experiments in the field.