Enforcing believability of computational models

The concept and approaches toward believability is quite broad. The goal is to propose a set of recommendation to designers. First propose a definition of ENACTIVE interface and then proposing a couple of examples.

Designing a believable interface

Depending of the approaches and the field, Four different approaches to the concept of believability have been proposed:

• Believability consists in the (accurate) perception of realism.
• Believability consists in the possibility of experiencing the perceptual properties of the virtual environment
• Believability consists into the similarity between the representation (algorithm device which implements the algorithm) and the represented condition (real world)
• Believability is the condition which is realized when the pattern of stimulation is experienced as an object and as objective

The four approaches can be unified into a common proposal of characterization of the notion of believability:

• Believability is a characteristic of those experiences in mediated conditions that present the same level of intersubjectivity and independence from the user (objectivity) of the interface as experiences with real objects of the real world do.
• In this sense, believable experiences with virtual objects resemble experiences with real objects. Nevertheless, believable experiences in mediated conditions can be very different and eventually totally new in respect to the experiences with real objects and the real world. It is not the simulation of real objects that makes a mediated experience believable, but the production of the conditions that make a pattern of stimulation an intersubjective and objective experience.
• The independence from the user cannot be understood as independence from his actions and performances. In fact, rather than the virtual objects and their resemblance with real objects, the notion of believability concerns a complex experience which includes the actions of the user during the interaction with the virtual objects and the appropriate algorithm and devices which implement the virtual object.
• The characteristics of the actions and performances that the user can display in the mediated condition hence play an important role in the judgment of believability.

Experimental investigations

Simulating reality with generic haptic devices

The experiment exploits an immersive helmet and the EPFL Haptic Workstation™ from Immersion Corporation (Figure 1b) If necessary it can be completed with the haptic-feedbacked steering-wheel we developed for driving lesson simulations. The set-up has first benefited from our prior work on gravity compensation to enhance the feasibility and the user comfort. We have used biofeedback techniques with an EMG to measure the muscular activity of the arms while changing the gears with different forces: only the static or with the dynamic ones. We also made some tests in a real situation to compare the graphs and the obtained results were satisfying. They show that our application can be considered as comfortable and believable after an adaptation time around 5 minutes. The main factors for maintaining the believability of the user experience were, in decreasing order:

• the haptic guidance reflecting the user’s past experience of using a gear box; it maintains a transparent mapping between the proposed gesture and past learned gestures. For example the users can easily shift the gears without looking at the lever because they can haptically feel the neutral point.
• The gear-shifting forces applied during the simulation are very close to the real ones except that we had to slightly scale them down due to the Haptic Workstation™ limitations (100N at the max).
• The simplified grasping of the lever does not need to be very accurate to trigger the believability of the gear shifting.

Figure 1: Virtual cockpit (a,c) with haptic feedback on the right wrist (b) for evaluating the believability of haptic manipulation of a gear box.

Virtual Reality Exposure Therapy

We have conducted a study over eight social phobic subjects. The aim of this study was to evaluate the flexibility and believability of virtual reality as a therapeutic tool in the confines of a social phobia behavioral therapeutic program. Its second goal was to use virtual exposure to objectively evaluate a specific parameter present in social phobia, namely eye contact avoidance, by using an eye-tracking system. Analysis of our results showed a tendency to improvement in the subjects’ feedback to specific assessment scales, which was correlated to the decrease of eye contact avoidance. The results showed that the presented virtual reality exposure therapy protocol could be successfully applied to social therapy.

Virtual Reality Exposure Therapy therefore seems promising. Since this study gave us some good results and showed visible improvements, we can conclude that our scenes were sufficiently believable in order to provoke anxiety in patients. They were believable enough for most of them (7 out of 8 patients) to project themselves into real life situations and feel, if only partially, the same anxiety as they would have in the corresponding real life situation.

Figure 2 (a) Eye tracking results before treatment (b) Eye tracking results after treatment

Enhancing believability through emotional behaviors

A very popular area of research today is the design of virtual characters that portray in one or multiple ways emotional behaviour. Emotion does not only influence our facial expressions: also body motions, speech and decision-making are affected. We are currently performing an evaluation study to find out how emotions expressed through the face and body are perceived by users. Figure 2 shows some frames taken from the videos to be used in the evaluation study. The goal of this study is firstly to determine how important face and body animations are when portraying emotions. A second goal is to find out in what situations body and/or face motions should be used to portray emotions and in what situations they are not necessary.

Figure 3. Some example postures and facial expressions used for the evaluation study.

The body animation system is based on the idle motion synthesizer described by Egges et al. The facial animation is handled by an MPEG-4 compliant face animation system. Evaluation of how emotion is perceived through body and facial animation will be performed by answering a questionnaire on a website. Several videos will be presented to illustrate each question asked to the subjects. Four sets of tests will be performed:

The first set of tests investigates if different emotions can be distinguished, showing only facial, body or both animations together.
The second set of tests estimates the degree of credibility of facial and/or body animation.
The third set of tests investigates how body animation influences the perception of an emotional state.
Finally, the last set of tests shows which emotion is perceived when different combination of emotional facial and body animation are played.

Sufficient components of conversational facial expressions:

This experiment is the first in a line of research aimed at producing an initial, systematic description of the parts of a face that are necessary and sufficient to not only recognize an expression, but also to perceive that expression's sincerity. To search for the facial regions that carry information about different expressions, we used a model-based image manipulation technique,) to systematically alter video sequences, where portions of a face in a video sequence were selectively "'frozen"' (i.e., replaced with a static photograph of that region).

Figure 4. Experimental design of FFS

This experiment used three tasks. The first task was to identify the expression. This was done by selecting the name of the expression from a list that was displayed on the side of the screen (see Figure 3). The list of choices included all from the actors recorded expressions as well as “none of the above”. The second task was to rate the believability of the expressions using a 7 point scale (where a rating of 1 indicated that the actor was clearly pretending and a value of 7 indicated that the actor really meant the underlying emotion). Finally, the participants were asked to rate the naturalness of the expression. Specifically, participants were asked to indicate if what they just saw is something that people normally do.

Overall, the expressions were rather easy to identify, even without a conversational context (between 70 and 100% accuracy, on average). Furthermore, most expressions seem to rely primarily on a single facial area to convey meaning, with different expressions using different facial areas. For example "'agree"' and "'disagree"' depend mainly on the rigid head motion, whereas for expressing happiness the mouth region carries most of the information. Although these results shed further light on what areas need to move to produce recognizable expressions, it still remains to be determined exactly how these areas move. For the thinking expression, for example, the eye motion is very important, but it seems to be rather irrelevant in which direction the gaze move.

Musical instrument believability: haptics and sounds cooperation

The two following experiments (Figure 4) show a minimal model, which allows to produce in real time believable glass-finger friction sounds and bowed strings sounds on all the range of the manipulation. There is a lot of acoustical models for this type of phenomena. They focus on the complex rosin material that regulates the sticking between the two objects. These models are very complex and they are used, as the physically realistic models of strings, to design real objects. Nevertheless, they have never been able to render all the sensible and complex modulations appearing during performance: timbre modulation, pizzicati, way of attacks, creaking, etc. A simple model as been designed, not focused on the realistic reproduction of the morphological properties of the objects themselves but on the interaction between the surface (resp. the string) and the hand, through a simple non-linear friction model modulated by velocity and pressure. The force feedback device is a 2D stick, 1D for the velocity and 1D for the pressure. It moves only on about 5 cm. The surface (resp. the string) is simply represented by a very few number of uni-dimensional masses linked by simple uni-dimensional visco-elastic constraints.

Figure 5 Minimal physically - based models for haptics-sounds Presence

The model focuses on the closed-loop between player and object, with a high quality of reactivity (less than 0,3ms between the device inputs and force outputs). Thus the morphology of the friction is completely different than in real situation but the basic features of dynamics of the interaction are rendered.

Several professional musicians and acousticians as well as several novice people who have tried the experiments, concluded unanimously to the believability of the “violin character”, pointing to the strong dynamic adapted coupling between this minimal physically - based instrument and the player as the critical parameter responsible for the presence of this virtual instrument.