A Software Framework for Multimodal Human-Computer Interaction Systems


Share/Save/Bookmark

Shen, Jie and Pantic, Maja (2009) A Software Framework for Multimodal Human-Computer Interaction Systems. In: IEEE International Conference on Systems, Man and Cybernetics, SMC'09, 11-14 Oct 2009, San Antonio, TX, USA.

[img]
Preview
PDF
340Kb
Abstract:This paper describes a software framework we designed and implemented for the development and research in the area of multimodal human-computer interface. The proposed framework is based on publish / subscribe architecture, which allows developers and researchers to conveniently configure, test and expand their system in a modular and incremental manner. In order to achieve reliable and efficient data transport between modules while still providing a high degree of system flexibility, the framework uses a shared-memory based data transport protocol for message delivery together with a TCP based system management protocol to maintain the integrity of system structure at runtime. The framework is delivered as a communication middleware, providing a basic system manager and well-documented, easy-to-use and open source C++ SDKs supporting both module development and server extension. The experimental comparison between the proposed framework and other similar tools available to the community indicates that our framework greatly outperforms the others in terms of average message latency, maximum data throughput and CPU consumption level, especially in heavy workload scenarios. To demonstrate the performance of our framework in real world applications, we have built a demo system which is used to detect faces and facial feature points in real-time captured video. The result shows our framework is capable of delivering some tens of megabytes of data per second effectively and efficiently even under tight resource constraint.
Item Type:Conference or Workshop Item
Copyright:© 2009 IEEE
Faculty:
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:
Link to this item:http://purl.utwente.nl/publications/69479
Official URL:http://dx.doi.org/10.1109/ICSMC.2009.5346338
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page