Multimodal Interaction Flow Representation for Ubiquitous Environments - MIF: A Case Study in Surgical Navigation Interface Design

Abstract

With the advent of technology, new interaction modalities became available which augmented the system interaction. Even though there are vast amount of applications for the ubiquitous devices like mobile agents, smart glasses and wearable technologies, many of them are hardly preferred by users. The success of those systems is highly dependent on the quality of the interaction design. Moreover, domain specific applications developed for these ubiquitous devices involve detailed domain knowledge which normally IT professionals do not have, which may involve a substantial lack of quality in the services provided. Hence, effective and high quality domain specific applications developed for these ubiquitous devices require significant collaboration of domain experts and IT professionals during the development process. Accordingly, tools to provide common communication medium between domain experts and IT professionals would provide necessary medium for communication. In this study, a new modelling tool for interaction design of ubiquitous devices like mobile agents, wearable devices is proposed which includes different interaction modalities. In order to better understand the effectiveness of this newly proposed design tool, an experimental study is conducted with 11 undergraduate students (novices) and 15 graduate students (experienced) of Computer Engineering Department for evaluating defect detection performance for the defects seeded into the interface design of a neuronavigation device. Results show that the defects were realized as more difficult for the novices and their performance was lower compared to experienced ones. Considering the defect types, wrong information and wrong button type of defects were recognized as more difficult. The results of this study aimed to provide insights for the system designers to better represent the interaction design details and to improve the communication level of IT professionals and the domain experts.