Multisensory information transfer

Experimental work has involved investigation of a navigation/recognition task and an assembly task, both previously developed within the ENACTIVE network for use as research tools. In one implementation of the navigation/recognition task, at ULUND, different strategies for navigation between targets within a non-visual virtual environment have been investigated. Various audio and/or force-feedback cues are provided to the experimental subject. Attractive forces which help the subject to “home in” on a target are found to be particularly effective.

Another implementation of the navigation/recognition task, at UNEXE, involves active exploration of a 2D virtual workspace, as shown in the images. The experimental subject is asked to discriminate virtual textures in an odd-one-out-from-three task.

Figure 1. Top: the tactile stimulator, which moves within a 2D workspace in response to the subject’s exploratory movements. Bottom: exploratory path – subjects were asked to explore the three targets, each 40 mm square.

Stimuli were specified in terms of mean amplitude, frequency content and spatial distribution of the vibrotactile stimulation. Perhaps the most interesting result from this experiment is a strong interaction between the perceived spatial aspects of the texture and the stimulation frequency. If the stimulation frequency is changed from 40 Hz to 320 Hz, the perceived sensation during active exploration changes much more if the texture is spatially non-uniform than if it is spatially uniform.

The assembly task involves insertion of two pins into two holes. The full scenario consists of a “pin” model composed of one circular and one square pin, a wall which contains an opening, a “hole” model which contains one circular and one square hole, and seven obstacles around the “hole” model. The experimental subject is required to pass the “pin” model through the opening in the wall and then insert the “pin” model into the “hole” model while avoiding the obstacles.

The platform at LABEIN is based on the GRAB device from PERCRO (see image). A new utility allows rotation of the held model by means of a combination of the keyboard and a switch on the GRAB. The user is assisted in the task by virtual guides: once the system has detected a common assembly axis (when the user is close enough) the movement of the user is constrained along the common assembly axis, allowing rotations only if there is any degree of freedom (as when only one circular pin is active), until the model is totally assembled.

Figure 2. The assembly task at LABEIN (left) and DLR (right)

The platform at DLR helps the user to undertake the task by displaying collision forces and torques from the virtual assembly scenario. These collision forces and torques are computed by a novel haptic algorithm and updated every millisecond. DLR use their light-weight robot as a 6DoF haptic device which, through an extremely light-weight design with a force-to-weight ratio of 1:1, is able to display forces and torques realistically. Thus, during the assembly simulation (see image), the user can translate and rotate the pin-object intuitively just by moving the control handle (see image).

In order to have a realistic transmission of the interaction forces during the reach and move phases, different possible mechanical connections between the finger and the interface device have been conceived and investigated by PERCRO. Several kinds of mechanical attachments for the GRAB system have been realised and tested (see image).

Figure 3. The three different solutions for connection to the fingertip