Cognitive-aware Modality Allocation in Intelligent Multimodal Information Presentation


Cao, Yujia and Theune, Mariët and Nijholt, Anton (2010) Cognitive-aware Modality Allocation in Intelligent Multimodal Information Presentation. In: Ling Shao & Caifeng Shan & Jiebo Luo & Minoru Etoh (Eds.), Multimedia Interaction and Intelligent User Interfaces: Principles, Methods and Applications. Advances in Pattern Recognition . Springer Verlag, London, pp. 61-83. ISBN 9781849965064

[img] PDF
Restricted to UT campus only

Abstract:Intelligent multimodal presentation (IMMP) systems are able to generate multimodal presentations adaptively, based on the run-time requirements of user-computer interaction. Modality allocation in IMMP system needs to adapt the modality choice to changes in various relevant factors, such as the type of the information to be conveyed, the presentation goal, the characteristics of the available modalities, the user profile, the condition of the environment and the type of user task. In this study, we emphasize that modality allocation in IMMP systems should also take into account the cognitive impacts of modality on human information processing. We first describe several modality-related cognitive and neuropsychological findings. Then a user study is presented to demonstrate the effects of modality on performance, cognitive load and stress, using a high-load and time-critical user task. Finally, we show a possible way to integrate relevant cognitive theories into a computational model that can systematically predict the suitability of a modality choice for a given presentation task.
Item Type:Book Section
Copyright:© 2010 Springer
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:
Link to this item:
Official URL:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page