Hardware and software trends and standards

Enactive interfaces consist of several devices, simulations and rendering loops. This produces a highly complex software environment including different computers, operating systems, man-machine interfaces (of different modalities) and networks. One goal is to standardize the interfacing between theses components and to develop a common scripting language to

• allow an independent development of these components
• ease the development of new enaction scenarios
• and to simplify the cooperative work among enactive partners

Viability testing of using CORBA for real-time application

CORBA has been evaluated for real-time 3D immersive applications such as the treatment of phobic patients through the interaction with virtual humans. A general overview of its integration within the software architecture of EPFL is provided in Figure 1. CORBA acts as a generic communication interface between interpreted Python scripts, on the application’s side, and on the other side the middleware VHD that integrates all the key features of such application (e.g. viewer, animation of virtual humans, etc…). Basically, the Python script is the “client”. In it, the user determines which operations one wants to undergo on a low-level application object. CORBA then allows requesting the VHD service through the IDL stub.

CORBA has been selected for this evaluation as it is a standard for distributed objects. It allows an application to request an operation to be performed by a distributed object and for the results of the operation to be returned back to the application making the request. The key features we have taken advantage of are:

• supports many existing languages: and supports mixing these languages within a single distributed application (e.g. C and Python in our case).
• supports both distribution and Object Orientation: exposing the interfaces of functions is really natural when programming with objects. The transparent call of functions between programs (through network or not) is definitely a useful feature.
• provides a high degree of interoperability: This insures that distributed objects built on top of different CORBA products can communicate (e.g. Windws and LINUX)
• Allows to split computationally heavy tasks (achieved in low level C components) from higher level tasks achieved in interpreted Python scripts (e.g. scenario management)
• The communication supports real-time constraints (when used carefully).
• Efficiency of the design loop as the main program (3D simulation) may keep running (avoiding long restart) while the user tries and restarts the clients (python scripts) several times

Figure 1: the use of interpreted Python scripts together with CORBA and the middleware VHD greatly eases the application design, especially for complex interactions in 3D environments such as those targeted by the EPFL-VRLAB (e.g. treatment of phobic patients through an immersive experience)

On the downside, we have experienced a few robustness issues when used within our software architecture (Figure 1):

• some maintenance has to be done regularly, such as file deletion or CORBA server shutdown and restart.
• the CORBA server can be capricious. Depending on the machine and the OS used, it is possible to have to start the server manually. By letting it start automatically, it can cause problems such as very slow execution.
• Since we are using an external server to execute scripts, execution is slightly slowed down ; this could be an issue for real-time critical applications. For less demanding applications such as ours it can lead to a slight decrease in the frame rate, hence a less comfortable experience of the end user.

In summary, CORBA is clearly a standard that helps the authors of complex 3D interaction applications to cut down on design iterations by hiding the low level software layers (no recompilation needed when used in conjunction with an interpreted language such as Python) and the nasty details of network programming. Its interoperability on Windows/LINUX platforms is a second important advantage in today’s mixed operating systems environments.

Haptic rendering libraries in general enactive framework

Enactive interfaces often make use of force feedback devices. Therefore, part of this research direction concerns the standardization of force feedback and haptic rendering. In this light, VRPN is investigated as this library provides a standardized way of accessing interaction devices. The support of force feedback has been limited in the past. During the Enactive project, the support of VRPN for haptic feedback has been extended. One of these extension efforts concerns the use of force fields in VRPN. Due to the distributed nature of VRPN, it is difficult to extend the functionality of the haptic rendering algorithms, as the VRPN server must be recompiled. In order to overcome this problem with respect to the use of force fields, scriptable force fields were introduced in VRPN. These force fields are defined in the Lua scripting language and are automatically executed by the extended VRPN force-feedback server, thereby avoiding deployment problems.

Figure 2: General integration of haptic rendering in an enactive VRPN framework

Another problem with haptic rendering is the fact several algorithms have been defined without a formal basis to compare existing algorithms with a newly developed algorithms. It is thus difficult to judge to validity of a new algorithm. Within the Enactive project, a method for objectively comparing haptic algorithms has been developed. In 2006, this method has further been refined.

Design of enactive applications over the Internet

Multiple Device Support in HapticWeb

The support of multiple haptic devices is one of the fundamental requirements for the spreading of a haptic application framework, and HapticWeb expands the variety of haptic devices supported by the CHAI 3D library with the support of networked devices and script defined devices. When no device is specified by the developer the platform tries to automatically identify the attached device and report its characteristics to the application, like workspace and maximum force. Finally if no device is attached a virtual device using a standard mouse or a 6-DOF mouse is provided.

Apart automatic identification HapticWeb provides the possibility of initializing different devices by using a device URI. A device URI specifies the type of device, its identification if there are many of them and optional parameters expressed using the query part of the URI. For example a PHANTOM can be accessed using ``phantom://default?mode=direct'' where the mode parameter specifies use of Direct I/O if available. In the case of a remote device we can write ``udp://145.22.33.44:1203/device0''. In general we suggest using the automatic device configuration because it reaches the maximum audience but in some cases a specific one could be required.

The HapticWeb platform allows also to define device filters that can be used for various purposes, like emulating device features, debugging or recording. A recorded haptic trajectory can then be played back using the ``null'' device.

Multiple devices can be instantiated as well, and each can be associated with a tool for the force rendering of the point of contact.

One of the problems in the portability of haptic applications is the difference between the devices in terms of workspace and force feedback performance; we address the problem by providing to the developer detailed information about the device and in particular allowing the scaling the workspace.

High Performance Haptic Scripting

The haptic loop is one of the key aspects of any haptic application because its efficient implementation provides a stable haptic interaction, usually obtained by maintaining a rate of 1KHz. The HapticWeb platform allows the developer to choose between two different approaches for the definition of the haptic loop. In the first approach the developer describes the haptic scene using a scene graph made of 3D models and haptic effects, and the engine manages the haptic loop in a way transparent to the program, eventually moving the Haptic thread to a secondary Cpu or computing core. The developer can access and manipulate the state of the objects only after obtaining a lock on the haptic scene. With this approach the resulting performance of the haptic loop is the same as of a native application, that is only limited by the precision of the timer's scheduling of the operating system.
The second approach is more flexible because it allows the developer to use both the haptic scene graph and scripting to compute the force feedback. The purpose of this feature is to allow experimentation with new haptic effects or rendering techniques not provided by HapticWeb. In general the performance difference between the scripting approach and the native haptic loop depends on the multithreading support of the scripting system.

Figure 3: Virtual Machine Concept in the HapticWeb approach

By default XVR is a multithreaded execution environment in which the virtual machine running the script is a shared resource associated only to one thread at a time. At each iteration on the two main loops, one for the graphics running at 75Hz and the other a generic timer running up to 1KHz, it waits for the virtual machine semaphore, allowing the execution of the corresponding callback function. In this way the scripting evaluation is equivalent to a single threaded approach that simplifies the synchronization aspect for the user, but can introduce performance problems in the haptic loop. The force computed by the scripted haptic loop is sent to a native thread that runs exactly at 1KHz and communicates with the device's driver. In this way the delay of the timer loop is a possible source of instability.
HapticWeb solves the multithreading problem with a two Virtual Machine approach. The main Virtual Machine has a script that controls the main application loop, the graphic rendering and the command to the other peripherals. The Haptic Loop is then implemented using the Secondary Virtual Machine that is isolated from the main and it runs in a separate thread eventually in separate core. Such approach allows a safe experimentation of new devices, new force fields or computation of the force feedback model. The communication between the two Virtual Machine is provided by a shared memory mechanism over which synchronized variables and message passing systems are provided. The Figure 3 shows the two Virtual Machine running the three main threads, and communicating each other using the shared memory.