Equivalent representations of multimodal user interfaces

Van Hees, Kris; Engelen, Jan
November 2013
Universal Access in the Information Society;Nov2013, Vol. 12 Issue 4, p339
Academic Journal
While providing non-visual access to graphical user interfaces has been a topic of research for over 20 years, blind users still face many obstacles when using computer systems. Furthermore, daily life has become more and more infused with devices that feature some kind of visual interface. Existing solutions for providing multimodal user interfaces that ensure accessibility are largely based on either graphical toolkit hooks, queries to the application and environment, scripting, model-driven user interface development or runtime adaptation. Parallel User Interface Rendering (PUIR) is a novel approach based on past and current research into accessibility, promoting the use of abstract user interface descriptions. Based on a single consistent conceptual model, PUIR provides a mechanism to render a user interface simultaneously in multiple modalities. Each representation of the UI provides equivalent semantics to ensure that collaboration between users of different modalities is facilitated. The possible application of this novel technique goes well beyond the realm of accessibility, as multimodal interfaces become more popular and even needed. The design presented here has been implemented as a prototype for testing and further research in this fascinating area of HCI.


Related Articles

  • Banking on Multimodality. YUSCHIK, MATTHEW // Speech Technology Magazine;Nov/Dec2012, Vol. 17 Issue 6, p5 

    The article focuses on multimodal user interface (MMUI), particularly for mobile banking. The rapid pace of the evolution of the smartphone is presented in relation to assessed mobile services needs. The benefits of multimodal mobile banking are discussed including its value drivers like...

  • ORGANIC USER INTERFACES: FRAMEWORK, INTERACTION MODEL AND DESIGN GUIDELINES. Zaki Ghalwash, Atef; Khaled Nabil, Sara // International Journal of Ad Hoc, Sensor & Ubiquitous Computing;Aug2013, Vol. 4 Issue 4, p39 

    Under the umbrella of Ubiquitous Computing, lies the fields of natural, organic and tangible user interfaces. Although some work have been made in organic user interface (OUI) design principles, no formalized framework have been set for OUIs and their interaction model, or design-specific...

  • Project Mudra: Personalization of Computers using Natural Interface. Dave, Ankit; Bhumkar, Yogesh; Abraham, Abin; Sugandhi, Rekha // International Journal of Computer Applications;09/15/2012, Vol. 54, p42 

    In recent years, the popularity for natural interfaces has experienced tremendous growth. Natural interfaces provide the computers or the so called machines with a sense of humanity. The basic knowledge required to understand and utilize a computer system is reduced to a minimal extent. There...

  • NELS MUSES….  // ABQ Correspondent;Jan2009, Issue 156, p2 

    This article focuses on the significance of the December 31,1969 5PM time/date stamp to the information technology industry. According to the author, Windows followed the lead of Unix for recording a time/date stamp. It is stated that Unix has the idea of "The Epoch": time in seconds since the...

  • DKBTrace.  // Network Dictionary;2007, p157 

    An encyclopedia entry about "DKBTrace" is presented. It was a graphical ray tracing program which was written by David K. Buck and Aaron Collins for UNIX. Its last version was built in 1989. The program ran via the command line since it had no graphical user interface. Some of the quadric shapes...

  • AutoMicromanager: A microscopy scripting toolkit for LABVIEW and other programming environments. Ashcroft, Brian Alan; Oosterkamp, Tjerk // Review of Scientific Instruments;Nov2010, Vol. 81 Issue 11, p113708 

    We present a scripting toolkit for the acquisition and analysis of a wide variety of imaging data by integrating the ease of use of various programming environments such as LABVIEW, IGOR PRO, MATLAB, SCILAB, and others. This toolkit is designed to allow the user to quickly program a variety of...

  • Get to Know the PowerShell ISE. Angelopoulos, Alex K. // Windows IT Pro;Jul2011, Vol. 17 Issue 7, p32 

    The article highlights the Windows PowerShell Integrated Scripting Environment (ISE), which is a combined shell, debugger and script editor that comes with PowerShell and is considered significantly undersea. The lack of knowledge about its capabilities is one of the reasons the PowerShell ISE...

  • A COGNITIVE BASED MODEL FOR DIALOGUE MANAGEMENT IN MULTIMODAL INTERACTION SYSTEMS. Prates, Jonathan S.; Rigo, Sandro J.; Costa, Cristiano A.; Barbosa, Jorge L. V. // Proceedings of the IADIS International Conference on WWW/Interne;2015, p111 

    Researchers, in Multimodal Interaction Systems, devote substantial effort to the integration of external stimulus and signal, to the internal representation of this information and to the response generation. Nevertheless, they focus less effort on how to integrate studies in Cognitive...

  • Multimodal Semantics Extraction from User-Generated Videos. Cricri, Francesco; Dabov, Kostadin; Roininen, Mikko J.; Mate, Sujeet; Curcio, Igor D. D.; Gabbouj, Moncef // Advances in Multimedia;2012, p1 

    User-generated video content has grown tremendously fast to the point of outpacing professional content creation. In this work we develop methods that analyze contextual information of multiple user-generated videos in order to obtain semantic information about public happenings (e.g., sport and...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics