GATE project

About GATE project

Graphics Accessible To Everyone, GATE, is a project aiming to achieve the following goals. First, development of utilities deployed for easy picture annotation. Second, provision of blind users with support for exploring (“viewing”) pictures. And finally, development of a system utilized for generating images by means of dialogue and enabling the blind to create some limited form of computer graphics. The project is currently being developed at the the Faculty of Informatics, Masaryk University Brno, Czech Republic. As regards the GATE system, let us briefly describe its basic modules and principles.

The ANNOTATOR module supports image navigation and is closely connected to the graphical ontology. Its basic task is to inform the user about the graphical content in a non-visual way. The GATE system provides two basic ways doing so – verbally and by means of sound (see demonstration). There are two basic tools supporting this communication: What-Where Language and Recursive Navigation Grid.

What-Where Language, WWL, is a simple fragment of English. Each sentence of this language has the form of WHAT is WHERE or WHERE is WHAT. It enables the user to ask simple questions about the objects in the scene and their position (e.g. “Where is the tower?“, “What is in the middle?“, “What is in the background?“).

Recursive Navigation Grid, RNG, represents the navigation backbone of the system, dividing the picture space into nine identical rectangular sectors analogously to the layout of numerical keys 1-9 of the numerical keyboard. Each sector is subdivided in the same way recursively. This enables the user to investigate a point or region with demanded precision and to carry out “zooming“.

Verbal Information Module, VIM, controls the verbal part of the dialogue including the WWL communication. Possible misunderstandings in the communication are solved by VIM by invoking dialogue repairing strategies.

Two basic strategies of retrieving information are supported, being represented by GUIDE and EXPLORER modules. The task of GUIDE is to provide verbal information, exploiting both the pieces of information obtained by tagging the picture and the pieces of information gained directly from the picture format. The module provides EXPLORER with relevant information and cooperates with VIM and RNG.

The communication of EXPLORER is not primarily verbal, but analogue. It is controlled by means of mouse, digitizer, or numerical keyboard. The output sound information is also primarily non-verbal. The RNG module is exploited for navigation. The pieces of information that are related to the place, object or rectangle pointed to are both verbal and non-verbal. This allows the user to perform a quick dynamic exploration of the non-annotated details of the picture. The information about the explored color is provided by a procedure which is based on a sound representation of colors. The basic idea assumes the sound information to be a combination of special sounds assigned to the primary colors of a suitable color model (based on the RGB color model). The boundary of the picture and its vicinity is signalized by a special sound.