Preliminary work for vocal and haptic navigation software for blind sailors
Abstract: This study aims at the conception of haptic and vocal navigation software that permits blind sailors to create and simulate ship itineraries. This question implies a problematic about the haptic strategies used by blind people to build their space representation when using maps. According to current theories, people without vision are able to construct cognitive maps of their environment but the lack of sight tends to lead them to build egocentric and sequential mental pictures of space. Nevertheless, exocentric and unified representations are more efficient. Can blind people be helped to construct more effective spatial pictures? Prior works have shown that strategies are the most important factors in spatial performance in large-scale space. To encode space in an efficient way, we made our subject use the cardinal points reference in small-scale space. During our case study, a compass establishes a frame of external cues. In this respect, we support the assumption that training based on systematic exocentric reference helps blind subjects to build unified space. At the same time, this training has led the blind sailor to change his haptic strategies to explore tactile maps and to perform better. This seems to modify his processing of space representation. Eventually, we would like to study the transfer between map representation and environment mobility. Our final point is about using strategy based on cardinal points and haptic virtual reality technologies to help the blind improve their spatial cognition.
Keywords: non-visual representation, space, haptic
exploration, strategy, virtual geographic map, blind sailors,
Correspondance: Mathieu Simonnet,
This paper describes how we are taking into account blind people’s spatial cognition to conceive simulation software for blind people’s navigation on a virtual sea (1-4). Different spatial theories accord various spatial capacities to blind people (4). We know, however, that “the main characteristic of spatial representations is that they involve the use of reference” (5). The lack of sight tends to lead to a body centered spatial frame, but maps are external reference frames (6). How can we make exocentric reference easier for blind people when they encode space?
“Search strategies in haptic exploration are related to encoding processes” (2). Therefore, this statement means that teaching blind people exocentric representation should help them use haptic exploration strategies more effectively. Exocentric strategies aim at locating object to object regardless of the person’s body position. This top-down reasoning appeals to the cognitive level in the first place and to the sensory-motor level in the second place.
As the north is the prior knowledge acquired by map-readers (7,8), we taught blind people that the concept of cardinal points is an absolute exocentric reference. Thus, we made them practice reasoning about the cardinal concept to build exocentric spatial representations. According to Vygotsky’s socio-cultural theory (1896-1934), psychological instruments like writing and maps are able to reorganize individual cognition. The compass is one of them. After testing our spatial reference task, we have experimented systematical training with a compass. The analysis of these results focuses on the haptic exploration strategies.
In the future, we shall consider the essential question of the interest of cardinal strategy for transferring spatial capacities between maps and environment. Our position is that virtual reality can help blind people to connect micro and macro scales in the same exocentric reference frame.
After a survey of debates about non-visual representation during the last century, we are going to emphasize the distinction between egocentric and exocentric reference frames related to the lack of vision.
How do blind people build efficient space representations? During the twentieth century, different theories tried to answer this question, and many controversies about the role of previous visual experience appeared. According to the inefficiency theory (9), blind people are able to build unified space representations only from simple forms or elements. The results of a wooden blocks recognizing task show that “touch alone is not as efficient in the perception of (…) complex tactual form relationships as touch aided by visual images” (10). The results of a second experiment about direction estimations in a triangle completion task leads us to conclude that kinesthetic cues were better able to perform when translated into visual images (10).
The difference and inefficiency theories disagree. Fletcher assumes that previous results come from a testing artifact (5). Other experiments showed no difference between blindfolded and congenitally blind adults when they were dealing with material that was not “optically familiar” (11). Eventually, “lack of vision slows down ontogenic spatial development…but does not prohibit it” (12).
To conclude, we have to remember that during a spatial inference task, congenitally blind people performed as well as blindfolded (13). So, the congenitally blind persons are able to construct spatial cognitive maps but this capacity develops more slowly. The question is how the lack of vision slows down the construction of exocentric representation.
From egocentric to exocentric reference
Exceptions notwithstanding, there is general under-standing that in an egocentric reference frame, locations are represented with respect to the particular perspective of a perceiver, whereas an allocentric reference frame locates points within a framework external to the holder of the representation and independent of his or her position (14).
Vision is the first perceptive modality of data concerning spatial environment (15). Vision gives simultaneously varied information about objects and their configuration in distant space. In addition to visual modality, haptic or tactile-kinesthetic modality informs well about spatial layout (16). The sequential characteristic of haptic modality leads blind people to encode an environment in successive reference to their own body before executing spatial inference between external objects. Nevertheless, vision is neither necessary nor sufficient for spatial coding (6).
According to the Piagetian theory (1), during their development children construct external reference frames from egocentric reference in coordinating vision and tactile perception. When using haptic modality, blind people have to remember the different segments of the object as a whole in short-term memory. This cognitive effort allows them to construct unified exocentric representation of space (17.). In this respect, a recent experiment shows that “allocentric (exocentric) relations can be accurately reported in all modalities […]” (18). Thus, spatial representation is not limited to any particular sensory modality, although the processing is probably faster with vision.
To conclude, not all blind people really build an external frame of reference but are able to do so. Differences in coding strategies are implicated more than their capacities of spatial perception (6).
Since the 1990s, researchers have correlated exploration patterns with the nature of non-visual spatial representation. Evidence of exocentric reference superiority leads us to use semantic representation to help blind people to improve encoding processes. We will present our hypothesis about the role of the compass on spatial representation.
The known strategies
Tellevik (2) first tested three patterns of non-visual exploration. In this task, blindfolded subjects had to find objects in a large-scale environment. Using “perimeter” patterns, the subjects explored the boundaries of given area. In “gridline” patterns, the subjects investigated internal elements of the area to learn their spatial relationship. With using “reference-point” patterns, the subjects relate their exploration to salient elements. The results show that search strategy in haptic exploration may be differentially related to encoding processes. With “perimeter” and “gridline” patterns it was more difficult for the blind to change their perspective than with “reference-point” strategies. This result shows that the latter pattern is more exocentric than the others. So, we think “gridline” patterns do not really give information about the relation between elements. Consequently, the “gridline” pattern is an egocentric strategy.
One year later, Hill (3) emphasized a lack in literature about object-to-object relationships. In a direction-estima-tion task about the explored environment, the results showed that “perimeter” pattern is a “self-to-object” strategy. Furthermore, an “object-to-object” pattern is identified and linked to distance between object reasoning. Eventually an efficient chronology of these patterns seems to involve “perimeter” and “object-to-object” strategies.
In this respect, Thinus-Blanc (4) studied the correlation between exploration patterns and spatial performance in locomotion and handling space. Subjects without vision have to detect the changes in a previously explored spatial layout. The same two types of patterns of explorations are found in small and large-scale space. On the one hand, “cyclic patterns consist in visiting a sequence of objects, with the same one beginning and ending the cycle”. On the other, “the back-and-forth pattern is characterized by repeated trajectories between two places”. In accordance with the O’Keefe and Nadel (7) theory, the similarity between the first type of strategy and route knowledge and the second type and map knowledge let us emphasize that “cyclic” pattern is an egocentric reference frame and “back-and-forth” pattern is an exocentric frame of reference. Here the results verify the superiority of exocentric reference frame.
To summarize the research, Ungar (19) carried out a literature survey of cognitive mapping without visual experience. A synthesis of non-visual exploration patterns identifies seven distinct exploration strategies or patterns: home base-to-object, perimeter, grid, cyclic, perimeter-to-object, back-and-forth, and object-to-object strategies, which are summarized in Table 1.
Recently, a doctoral thesis about “perception and cognition of space by individuals who are blind or have low vision” introduced a new strategy called “perimeter-to-center” (20). The subjects explored a constructed maze, and located and remembered the positions of six different salient
Table 1. Ungar (19) modified table: Nature of strategies identified in Hill et al (3); Thinus-Blanc (4); Schinazi (20)
Home base-to-object (3)
Moving repeatedly between the home base (origin point for exploration) and all the others in turn
Explored the boundaries of an area to identify the area's shape, size and key features around its perimeter, by walking along the edge of the layout
Investigated the internal elements of an area to learn their spatial relationships, by taking straight-line paths from one side of the layout to the other.
Each of the four objects visited in turn, and then returning to the first object
Perimeter to center (20)
explored the boundaries to identify the area’s shape, size and key features around the perimeter and then inside of it
Perimeter to object (3)
Moving repeatedly between an object and the perimeter
Moving repeatedly between two objects
Object to object (3)
Moving repeatedly from one object to another, or feeling the relationship between objects using hand or cane.
points. The results emphasize two egocentric strategies: “grid” and “perimeter-to-center”. The first strategy consists in exploring the boundaries to identify the shape, size, and key features of the area around the perimeter and then the inside of it. We have added this egocentric strategy in Table 1.
All these strategies come from movement observations. The results show that blind people performed better when they used exocentric patterns. This evidence proves the positive correlation between a higher cognitive spatial level and exocentric strategies.
The cardinal strategy: a top-down process
A coherent relation between mental representation and sensorial information provides a semantic encoding. Thus, the exocentric or egocentric nature of previous spatial strategies results from cognitive processes. The top-down process, from the map concept to environment stimuli and the bottom-up process tightly fit into each other.
Subjects without vision have the same stimuli at their disposal. As they do not perform similarly, blind subjects probably do not use the same mental space concept. How can we induce blind people to use maps as representations?
Cognitive mapping processing requires external cues in long-term memory. We know that “the fact that the information which is reliably available in long-term prior experience influences modes of coding explains coding in blind conditions” (6). As we have already seen, the cognition of the north is one of the key prerequisites to reading a map.
Why not teach blind people cardinal points concept?
Acredolo et al. (21) explain, “information related to the immediate goal of an action is remembered more effectively than is information that is not” (2). In this respect, we ask our blind subject to remember spatial layout using cardinal points reference. This learning requires the use of a tactile compass to provide salient external cues. We conducted an exploratory experimentation to evaluate the efficiency of cardinal reference in space encoding.
As we have already seen, egocentric and exocentric spatial representations exist. This experiment attempts to observe if the compass leads to the use of haptic exocentric patterns of exploration. At the moment, the subject of our exploratory experimentation is an adventitiously blind individual. The man who is 45 years old, lost his sight at the age of 22. He agreed to be the first to test the following protocol.
The spatial task consists in reproducing a small-scale spatial layout in an absolute reference after changing position around a table.
Situation. Three square metal sheets are
placed at three different points, 90° rotated around the table. The sheets are
twenty-five centimetres wide. We use six magnetic pieces of various geometric
shapes such as a triangle, a cross, a trapezium, a disk, a half-disk
and a square. Each piece is covered with different textures: soft, rough, wire netting, cardboard, tactile lines and crossed tactile lines. All this aims at helping the subject to distinguish all these different objects from each other. The pieces are placed only on the first sheet; on the other two, they are placed next to them. A non-tactile gridline is drawn on this handling space. Each magnetic piece is placed in the middle of a five centimeter non-tactile square. The grid lines allow us to measure errors when the blind subject reproduces the layout (cf. Figure 1).
Subject’s experiemental activity. The subject sits at the round table and listens to the instructions. After haptically exploring the layout of the six elements on the first sheet without any time limit, the subject has to reproduce the first configuration on the two other empty sheets. The main point is that this spatial layout reproduction has to be in reference to absolute space and not to body position. So, after exploring the first board with the pieces, the subject rotates 90° round the table and manually reproduces the configuration on the second board. He does the task twice.
· Collection of results. From the beginning, the subject knows that results depend on the correct positioning of the pieces on the grid drawn on the sheet. The further the magnetic piece is situated away from the correct position, the more important will be the mistake. On the one hand, the tasks are videotaped to observe the subject’s haptic exploration strategies. Visualization allows us to identify the different haptic exploratory patterns the subject uses. On the other, to try to study the cognition of the subject, the subject is asked to verbalize his reasoning.
The interpretation of this experiment consists in comparing the differences between performance before and after learning the cardinal points. This experiment aims at evaluating the impact of our cardinal strategy in training.
The cardinal training consists of three training sessions. All the while the subject could use a tactile compass. We continue the training until the subject is successful.
· Task one. Cardinal Orientation question. The cardinal orientation question task is composed of two parts. The instructor places one magnetic piece and asks him to tell the relevant cardinal orientation between the piece and the centre of the sheet. Then the instructor asks questions about the cardinal orientation between two objects placed randomly on the table. After each answer, the subject is receives feedback from the instructor. The answers can be north, south, east, and west; northwest, southeast or north-northwest, east-southeast… After one correct answer, the subject stands up and walks a 90° rotation before sitting down in front of the next sheet. The instructor questions him about the cardinal orientation of another magnetic piece and stops after three corrects consecutive answers. At the end of this task, we can assume that the subject has internalised cardinal map representation.
· Task two: Cardinal Orientation positionning. This task consists in positioning elements around the center according to the instructor’s request. In the second part of the task, the subject is asked to place two objects on a cardinal axis such as southeast northwest for example. At the end of this task, we assume that the subject is able to apply his cardinal map representation to the physical environment. Thus the subject can now make use of cardinal orientation positioning.
· Task three: spatial layout production. The subject puts the six pieces wherever he likes on one sheet. Afterwards he has to do it again on the other two sheets. In this final task, we attempt to enable the subject to get into the habit of building his own favorite constants in the exocentric cardinal reference frame. For example, using the northwesterly corner as a reference point seems to be efficient.
When the previous three tasks are successfully completed, we have to wait one week before asking our subject to perform the reference task anew to avoid the straight recall effect of learning (22).
Before cardinal training, the subject had made seven mistakes. From a strategy point of view, on the one hand we clearly identified “home-base-to-object”, “cyclic” and “grid” egocentric patterns of haptic exploration and on the other, the “perimeter-to-object” exocentric strategy appears. In other words, the subject uses mostly egocentric spatial representation. Certain other behavioral cues emphasized this assumption. The video recording showed egocentric behavior during the spatial layout reproduction task before cardinal learning. The subject tried to turn the board in front of him before performing. As this was not allowed, he first used body references and put the pieces in wrong squares and then changed their position along with a slight body contortion. This performance allows us to think that the subject was still using a body referent frame.
After cardinal training, the subject made no mistakes. He used three exocentric haptic exploration patterns: “peri-meter to object”, “back-and-forth”, and “object to object”; and only one egocentric strategy: “home base-to-object” haptic pattern.
After this cardinal points training, we observed the subject’s exocentric behavior. Firstly, because he thought aloud we were able to examine a part of his spatial cognitive process based on the cardinal orientation. As wehad expected, the subject spoke only about cardinal orientation. For example, he said that the triangle was in “the northwesterly corner” instead of “the top left corner”. Secondly, he placed objects straight in the right position without body contortion. He seemed to make mental rotations in an easier way. Consequently, this explanation allows us to think that the subject encoded the location of the pieces in the spatial layout using an exocentric reference frame. We found these results, although based on only one subject, very revealing.
According to the previous non-visual spatial theories (3,4), exocentric reference provides a higher spatial cognitive level. The cardinal concept seems to put the blind at an even more superior spatial cognitive level. As we have already seen, the compass provides available external cues (8), regardless of the subject’s body position. We noticed that the subject used the tactile compass only during the first three minutes of the cardinal training but kept answering questions about the cardinal orientations. Moreover, the reaction times of the answers decreased as the training went on. This evidence supports the assumption that the subject succeeded in internalizing cardinal points in a map representation. In accordance to Vygotsky’s theory, a tactile compass, as psychological instrument, reorganizes spatial cognition for our subject.
Eventually, we may take patterns on Thinus-Blanc’s model of “two level spatial processing” to provide an explanation of spatial cognition. Our subject first used simple means to encode information to get acquainted with the environment. Consequently, the position of the north has been encoded from body reference. To learn the other cardinal points, however, specific maps are needed. The internalization of the relations between the north indicated by the compass and the different cardinal points proves the validity of exocentric organization as a context situated representation of the space.
We do not have the ambition to explain the general spatial cognition of the blind. Our point is to understand how tactile compass must be used to afford exocentric reference frame (Figure 2). The previous theories and results lead us
to think that our subject first touched the compass with ego-centric haptic patterns  to encode the north direction in a body reference frame . After this, he associated egocentric north with exocentric cardinal map  stocked in long-term memory. Then he was able to use haptic exocentric strategies  to encode spatial relationships between elements  in a situated cardinal representation  on a cognitive map. The evidence collected in this experiment supports the assumption that the use of a compass solicits exocentric patterns of exploration.
We have reservations about our conclusion because of our population characteristics. Our single subject is adventitioulsy blind and is familiar with compasses, maps, and sailing.On the contrary, the congenitally or adventitiously blind often have very little experience of maps and compasses. Thus, our conclusion remains a hypothesis. However, we are currently conducting this experiment with 20 blind people, including an experimental and a control group. Before concluding this experiment, we will emphasize that this study is a preliminary work for a more ambitious future project financed by CECIAA enterprize in CERV. In fact, our aim is to understand better the spatial cognition of the blind to create spatial virtual reality navigation tools for them. Our experiment remains in manipulatory space, however, questions about transfers between maps and largescale environment are involved (23). Can cardinal strategy training help blind people to improve their spatial autonomy? How relevant is cardinal strategy for us to conceive our haptic and vocal maritime software?
Usually, spatial representations can be indirectly built by symbolic media such as cartographic maps (24). Sailboat orientation is not conceivable without maps and compasses. Even maritime spatial representation of sighted people is necessarily organized with psychological instruments.
One particular spatial feature of sailing consists of tacking when the destination is in front of the wind. When sailors zigzag, they do not follow a route but realize spatial inference tasks. In other words, if they want to reach point A, they first sail toward an imaginary point B and wait a short time and turn. This is a very a difficult situation to explain orally while the boat is heeling. Moreover, if the crew encounter rocks in their path, blind people can no longer remain at the helm. Today, accurate information is available by the means of GPS (Global positioning system). However, map knowledge is required if the sailor wants to control his voyage, coordinates, bearings, distances, and waypoints. In this respect, we are devoting our work to create cartographic software that will enable blind people to learn mapping and prepare trajectories.
“Most users would prefer to access tactile maps at home” (25), that’s why we are setting up cartographic sailing simulators for blind sailors. They will be able to sail virtually with cartographic and wind constraints. Wind element and sailing principles are not complex, but it is more difficult to use them in egocentric spatial represent-ations. Our first step will be to find an easier way to teach maritime mapping—and not only maritime routing.
For a long time, sailors have employed cardinal references to find their way on the sea. That is why we think that our previous cardinal training task may be revised and reinvested in this project. The simplest means to test cardinal strategy and haptic exocentric patterns of exploration is to introduce a haptic device in this cartographic software. A haptic device is a “mechanical system that senses forces in remote environments and delivers those forces to the hand of the user in the form of a haptic display accessed via a rigid link” (26). Phantom is a cheap available haptic device. Regarding spatial maritime layout, we will mix haptic object identification and cardinal vocal announce-ments. For example, blind sailors will touch a buoy and automatically hear its name. After this, if blind sailors click with the Phantom on another object, the announcement of cardinal orientation between these two points will be vocally announced. This is the back-and-forth haptic exocentric strategy (4) of exploration using cardinal reference. We hope blind sailors will develop new efficient strategies on this virtual sea. We conclude maps would be better served if used in conjunction with other multimodal devices that provide alternative sensory inputs.
Next summer, our experimentation will begin. Blind people will explore a virtual map of “the Rade of Brest” by touch. Another purpose is to find how to represent the different elements of navigation charts more intuitively. The touch of the sea will be soft and smooth, the earth will be rugged and in relief, the sailboat will be a mobile triangle, the depth will speak when you click on it, etc… Only blind sailors will tell us what works and what does not. Eventually both sighted and blind people will be able to dream together about feeling the ocean currents, the movements caused by the swell, and one day perhaps in this virtual environment we will all be able to touch a shoal of fish swimming sixty feet under the boat!
Inhelder B. La représentation de l’espace chez l’enfant.
Tellevik JM. Influence of spatial exploration patterns on
cognitive mapping by blindfolded sighted persons. J
Hill EW, Rieser JJ, Hill MM, Halpin J. How persons with visual impairments explore novel spaces:
Strategies of good and poor performers. J
Thinus-Blanc C, Gaunet F. Representation of space in
Blind persons: Vision as a spatial sense. J
Fletcher JF. Spatial representations in blind children.
Development compared to sighted children. J
Understanding and representing space: theory and evidence from studies with
blind and sighted children.
O’Keefe J, Nadel L. The hippocampus as a cognitive map.
Understanding and learning maps. In: Kitchin RM, Freundschuh S, eds. Cognitive
mapping: Past, present and future.
Revesz G. Psychology and art of the blind.
10. Worchel P. Space perception and orientation in the blind. Psychol Monogr 1951;65(15):332.
11. Juurmaa J. An analysis of the components of orienta-tion abilityand mental manipulation of spatial relation-ships. Helsinski, SF: Inst Occupat Health, 1965.
Kitchin RM, Jacobson RD. Techniques to collect and
analyze the cognitive map knowledge of persons with visual impairment or
blindness: Issues of validity. J
13. Rieser JJ, Guth DA, Hill EW. Sensitivity to perspective structure while walking without vision. Perception 1986;15(2):173-88.
Klatzky RL. Allocentric and egocentric spatial
representations: Definitions, distinctions, and inter-connections. In: Freksa
C, Habel C, Wender KF, eds. Spatial cognition. An interdisciplinary approach to
representation and processing of spatial knowledge (Lecture Notes in Artificial
Psychologie cognitive de la cécité précoce.
Hatwell Y, Streri A, Gentaz E. Toucher pour connaître.
18. Carreiras M, Codina B. Spatial cognition of blind and sighted: Visual and amodal hypothesis. Cahier de Psychologie Cognitive 1992;12(1):51-78.
Ungar S. Cognitive
mapping without visual experience. In: Kitchin RM, Freundschuh S, eds.
Cognitive mapping: Past, present and future.
Schinazi VR. Spatial representation and low vision: two
studies on the content, accuracy, and utility of mental representations. CASA
Working Papers, no. 93.
21. Acredolo LP, Pick HL, Olson MG. Environmental differentiation and familiarity as determinants of children memory for spatial location. Dev Psychol 1975;50:1062-70.
22. Schmidt RA. A schema theory of discrete motor skill learning. Psychol Rev 1975;82:225-60.
23. Lahav O, Mioduser D. Blind persons’ acquisition of spatial cognitive mapping and orientation skills supported by virtual environment. Int J Disabil Hum Dev 2005;4(3):231-7.
24. Richardson AE, Montello D, Hegarty M. Spatial knowledge acquisition from maps, and from naviga-tion in real and virtual environments. Mem Cogn 1999;27:741–50.
Rowell J, Ungar S. Feeling our way: tactile map user
requirements—a survey. Int Cartogr Conf,
26. Lederman SJ, Klatzky RL. Haptic identification of common objects: Effects of constraining the manual exploration process. Percept Psychophysics 2004;66: 618-28.