Effective Haptic Rendering Method for Complex Interactions

The development of haptic technology is allowing the introduction of Virtual Reality systems as teaching and working tools into many fields such as engineering (Howard & Vance, 2007; Savall et al., 2002) or surgery (Basdogan et al., 2004; Li & Liu, 2006). Haptic devices allow users to interact with a certain environment, either remote or virtual, through the sense of touch, considerably enhancing interactivity. A haptic device is a mechanism that allows users to control the movements of a virtual tool or a real robot and receive tactile and kinesthetic information from the working environment (Fig. 1). The usability of these systems is conditioned by the quality of the haptic feedback applied to the user. Technologically, the computation of appropriate and realistic haptic stimuli continues to be a complicated issue. The human sensory-motor system demands a fast update rate (at least 1 kHz) for the haptic stimuli applied to the user in order to avoid instabilities in


Introduction
The development of haptic technology is allowing the introduction of Virtual Reality systems as teaching and working tools into many fields such as engineering (Howard & Vance, 2007;Savall et al., 2002) or surgery (Basdogan et al., 2004;Li & Liu, 2006). Haptic devices allow users to interact with a certain environment, either remote or virtual, through the sense of touch, considerably enhancing interactivity. A haptic device is a mechanism that allows users to control the movements of a virtual tool or a real robot and receive tactile and kinesthetic information from the working environment (Fig. 1). The usability of these systems is conditioned by the quality of the haptic feedback applied to the user. Technologically, the computation of appropriate and realistic haptic stimuli continues to be a complicated issue. The human sensory-motor system demands a fast update rate (at least 1 kHz) for the haptic stimuli applied to the user in order to avoid instabilities in www.intechopen.com the system and to present rigid objects with reasonable stiffness (Shimoga, 1992). However, this update rate is often difficult to reach by haptic rendering methods, especially when working in complex environments. One possible solution is to reduce the computational cost of calculating the haptic response by decreasing the accuracy of the method. However, this can result in the emergence of discontinuities in the response. This leads to find a trade-off between the accuracy of the method, which guarantees a smooth and stable haptic response, and the computational cost. This chapter describes a haptic rendering method to properly compute interacting forces and torques in complex environments, ensuring improved feedback by seeking a compromise between continuity and computational cost. In addition, the proposed method pays particular attention to provide users with comfortable interaction. The method is valid for applications in which the virtual environment is composed of rigid and static objects, excluding deformable objects. The remainder of this chapter is organized as follows: Section 2 presents an overview of the related research on the area. Afterwards, Section 3 describes the haptic rendering method proposed by the authors, describing in detail all algorithms necessary to render appropriate and stable forces and torques to the user. The proposed method is then evaluated in Section 4 within two different virtual scenarios simulating common collisions during aeronautic maintainability tasks. Aeronautic virtual mock-ups have been selected for algorithms testing due to their high interaction complexity. Finally, conclusions and future directions are drawn in Section 5.

Related research
The process of computing and generating forces in response to user interactions with virtual objects is known as haptic rendering . The application of haptic rendering algorithms to complex contact scenarios becomes a challenging issue, due to the inherent cost of collision detection that induces slow force updates. The haptic display of virtual objects has been an active area of research throughout the last decade. Previous research in haptic rendering can be mainly classified within two groups: penalty and constraint-based methods.
When it comes to penalty methods, collision response is computed as a function of object separation or penetration depth. McNeely et al. (1999) proposed point-voxel sampling, a discretized approximation technique for contact queries that generates points on moving objects and voxels on static geometry. This approximation algorithm offers run-time performance independent of the environment's input size by sampling the object geometry at a resolution that the given processor can handle. Renz et al. (2001) adapted this method with several modifications for smoother and more stable haptic feedback. In another research project, Gregory et al. (2000) presented a 6-DOF haptic rendering system that combined collision detection based on convex decomposition of polygonal models, predictive estimation of penetration depth and force and torque interpolation. Kim et al. (2003) attempted to increase the stability of force feedback by using contact clustering, but their algorithm for contact queries suffers from the same computational complexity. Otaduy and Lin (2003) have presented a sensation preserving simplification technique for 6-DOF haptic rendering of complex polygonal models by adaptively selecting contact resolutions. Later, they have also presented a modular algorithm for 6-DOF haptic rendering, that provides transparent manipulation of rigid models with a high polygon count (Otaduy & Lin, 2006). Unlike penalty methods, constraint-based methods do not use the interpenetration between rigid objects to calculate collision response. These methods use virtual coupling techniques (Colgate et al., 1995) and restrict the movement of virtual objects on the surface of obstacles. Zilles and Salisbury (1995) proposed a constraint-based method for 3-DOF haptic rendering of generic polygonal objects. They introduced the "god-object", an idealized representation of the position of the haptic device that is constrained on the surface of obstacles. At each time step, the location of the god-object minimizes the distance to the haptic device and the difference between the two positions provides the force direction. Ruspini et al. (1997) extended this approach by replacing the god-object with a small sphere as well as proposing methods to smooth the object surface and add friction. Later, Ortega (2006) extended the 3-DOF constraint-based method of Zilles and Salisbury by employing a 6-DOF god-object. The different haptic rendering methods described above have contributed extensively to a better representation of contact events between virtual objects. However, haptic interactions with multiple contacts, which also include geometrical discontinuities, have not yet been adequately accomplished computing unrealistic or unstable haptic feedback in these cases. These type of situations are very common in real scenarios and therefore it is necessary to compute properly a stable haptic response in order to improve the usability of these systems. The proposed haptic rendering method overcomes limitations from previous approaches in this type of collisions.

Proposed haptic rendering method
The haptic rendering method outlined in this chapter computes the force and torque that result when a collision between two type of objects occurs: a virtual tool (mobile object) manipulated by the user of the haptic device and any object in the simulation (static object). Three main modules can be identified in the haptic rendering method proposed ( Fig. 2): collision detection, collision response and control module. The complete haptic rendering sequence could be described as follows: firstly, the control module acquires the position (U h ) and orientation (R h )o ft h eh a p t i cd e v i c ea n ds e n d si t to the collision detection module. With this information, the module checks for collisions between the mobile object and the static environment. If no collisions occur, it waits for new information from the control module. Otherwise, when a collision event occurs, the contact information of both static and mobile objects (C s , C m ) is sent to the collision response module which calculates the interaction force and torque. This haptic feedback approximates the contact force and torque that would arise during contact between real objects (F r , T r ). Finally, the collision response module sends this information to the control module, which applies it to the haptic device (F h , T h ), maintaining a stable system behaviour. A more complete description of each module can be found in the following sections.

Collision detection
The collision detection method presented in this chapter can handle non-convex objects without modifying the original mesh. A technique based on a spatial partition (voxels) has been chosen. Hierarchical methods like octrees have also been tested since they require less  (Borro et al., 2004), hashing techniques can solve the first problem by reducing memory storage up to 60 % without performance loss, and the choice of an optimal voxel size can be solved by means of an analytical solution based on the algorithm cost function. The method, in a pre-process, computes a voxel partition from the virtual scene and assigns each triangle of the static environment to its corresponding voxel. This voxel partition is used only for the static object. In addition, each voxel will have a flag identifying it as internal (V int ), external (V ext )o rb o u n d a r y( V bnd ). The last ones contain the triangles that define the surface of the static object. Next, at runtime, the partition model is used to detect the set of voxels in collision with the mobile object and to carry out interference checks between triangles. Fig. 3a shows an example of colliding static triangles (CST) and colliding mobile triangles (CMT) detected by the method in a virtual collision of a tool with an obstacle. As it can be seen colliding triangles do not provide enough information to delimit the volume that defines the intersection between the objects. Therefore, unlike other existing methods in the literature, the proposed algorithm detects additional triangles in order to calculate the intersection volume correctly (Fig. 3b). The union of the colliding triangles and these additional triangles are referred to as contact triangles.
The additional triangles can be classified within three groups: internal mobile (IMT), boundary mobile (BMT) and boundary static (BST). With regard to the mobile object, (IMT) are those in contact with V int whereas (BMT) are those in contact with V bnd , but that do not intersect with static triangles (CST) (Fig. 4). Regarding the static object, BST set of triangles are those in contact with V bnd (Fig. 5), but that do not intersect with mobile triangles. These triangles are usually in the centre of the contact area surrounded by colliding static triangles.

Mobile triangles not detected
Intersection volume At the end, the collision detection module yields two sets of contact triangles, one belonging to the static object (C s ) and the other to the mobile object (C m ).

Collision response
The algorithm proposed follows the well-known penalty methods based on elastic model, thus the haptic response that users feel as a consequence of a collision in the virtual environment is determined by a direction and a penetration value. Both factors have substantial influence on user perception. The force (F r )andtorque(T r ) are calculated as follows: where K is virtual stiffness, d is penetration depth, n force direction, cp the contact point and gc the centre of mass of the virtual tool.
In complex scenarios, it is common to encounter multiple contacts. The lists of triangles (C s and C m ) obtained by the collision detection module do not give a priory information about the number of different contacts. Therefore, the method divides these sets of triangles into different contact areas considering their spatial proximity. The forces and torques of each area are then added up to compute the net force and torque. A certain amount of static triangles form a contact area when they are adjacent. In other words, they share at least one edge. Once the different contact areas have been calculated from C s , it is necessary to identify the mobile triangles in contact that are associated with those areas.
To facilitate this process, a sphere is created in each contact area that covers the bounding box defined by its triangles. Finally, each triangle of C m is associated with the contact area depending on the sphere which contains it. Fig. 6 shows an example of the division of contact areas.
The difficulty in calculating an appropriate and stable haptic response increases when the geometry has sharp edges, since haptic instabilities often appear due to abrupt force direction or penetration depth changes. In order to detect these cases, each contact area is sub-divided into different contact zones that provide information about the nature of the geometry in collision. The collision response module subdivides a collision area into different contact zones taking surface C 1 discontinuities into account. When two triangles share an edge and the angle between their normal vectors is lower than a fixed value, the edge is designated as "smooth". Triangles in a contact zone must be interconnected and all the shared edges must be smooth. There will be as many contact zones as necessary to satisfy the condition of smooth connectivity (Fig. 7). Once the contact areas and zones have been detected, the method computes the contact normal vector (force direction) of each area (n c ). If the contact area has a unique contact zone, the contact normal is computed as the normalized sum of all normal vectors of the static triangles of that zone. Otherwise, when a contact area has two or more contact zones, the  7. Example of two contact areas. The second one has two contact zones contact has occurred in an area of the static object that is not a continuous surface. In this case, static triangles do not provide enough information to obtain a suitable force direction. Therefore, n c is computed as the normalized sum of all normal vectors of the mobile triangles of that area. This solution enables smoother direction transitions when interacting with sharp edges.
After computing the force direction in each area, the following step is the calculation of the minimum distance required to separate the two objects, known as penetration depth (d). For that purpose, the method samples the volume of intersection measuring heights throughout this volume to determine the penetration between two objects. These heights are determined by tracing a ray from the centroid of each static triangle in n c direction (Fig. 8). If this ray intersects with a mobile triangle (Möller & Trumbore, 1997), the height is defined as the

Contact mobile triangles
Contact static triangles Heights The last parameter necessary to compute collision torque is the contact point (cp). Selecting an inappropriate point such as the most penetrating one for each contact area might lead to non-continuous changes in the haptic feedback (Hasegawa & Sato, 2004). To avoid this effect, the proposed solution is to choose a representative contact point for each contact area. This point is calculated as the average of the midpoints of the boundary voxels associated with each contact area.

Control module
The control module receives the ideal interaction force and torque calculated by the collision response method, adapts them to the device's capabilities and applies them to the user at a 1 kHz sampling rate. Since the control loop runs faster than the collision module, several strategies must be implemented in order to avoid abrupt changes in contact force and torque when collision information is not available (Savall et al. (2002)). There are two main problems that should be taken into account. The first one is the delay that exists in the collision-related information. This information, calculated by the response module, is valid for a previous user position, but not the actual one. The second problem resides in the existence of some control loops without collision-related information, since the response module is slower than the control loop. In order to deal with these problems, a strategy based on intermediate representations (Adachi et al. (1995)) is implemented. Let M h =(F h , T h ) be the six-dimensional vector of the force and torque that we want to apply to the user and X h =(U h , R h ) be the six-dimensional configuration of the haptic device (U h represents the position, and R h the axis angle representation of the rotation). First, the control method updates the force and torque (M r ) computed by the response module at a previous sampling period j, to the current one i (K represents virtual stiffness): These forces and torques might vary quite abruptly if they are applied to the user every time they are updated by the collision response method. Therefore, the control module restores In the next sampling period (i + 1), the method also takes the new movements performed by the user into account: This will continue until the collision response method updates the collision-related information. To determine n, the optimal value should approximate the number of control sampling periods that the collision module needs to compute the response. Since this number can vary significantly depending on the number of triangles in collision, a conservative number can be set. However, if n is very high, the method may excessively filter the signal. Moreover, n can have a fixed value during the entire task or can be modified by means of the average collision response delay in previous sampling periods.

Implementation and results
Two different virtual scenarios have been designed to test the effectiveness and stability of the proposed haptic rendering algorithm on complex interactions. A 3-DOF torque feedback device has been used for the experiments. The mechanism was designed and built at CEIT and inspired by past research (Angerilli et al., 2001). Fig. 9 shows the device and its main specifications. The system is controlled by a dSPACE DS1104 board that reads encoder information, processes the haptic control loop and outputs torque commands to the motors. Graphic rendering and

Analysis of multiple contacts and geometrical discontinuities
The first scenario is composed of two parallelepipeds, one central cylinder and a virtual tool similar to a clamp (see top of Fig. 10). The virtual scenario consists of 25, 000 triangles while the clamp is composed of 1, 500 triangles. The aim is to analyze the behavior of the method when multiple contacts and geometrical discontinuities are involved. For that purpose, a sequence of different collisions has been simulated: 1. Collision with the central cylinder in which two contact areas are involved.
2. Collision with the corners of the two parallelepipeds (C 1 discontinuities).
3. Collision with four contacts simultaneously, combining the previous cases. Fig. 10 shows the torque feedback computed in all three axis by the collision response method during the sequence. The figure also shows the number of contact triangles detected and the computational cost in each frame. Notice in the figure that the torque feedback applied to the user during collision is quite smooth and does not offer abrupt changes or discontinuities that may degrade the user's perception of contact. The last figure shows that the collision response method computes the haptic feedback at an average of 2.5 ms, which is not far from the optimal computation time (1 ms) necessary for a realisic haptic experience. This allows the control module to compute therealforcesandtorquesat1kHzusingalownumberofn transitions (described in Section 3.3), and thereby maintaining a stable system behaviour. Specifically, for this experiment, the number of transitions n for the control algorithm was set to 5.

Simulation of a disassembly task
The task designed for the second experiment is similar to the extraction of a clamp from a pipe, which frequently appears in engine disassembly tasks in aeronautics maintenance. Once the clamp is unfastened, the exit path is established by following the spatial trajectory laid out by the pipe itself. Along said path, the curves and bends of the pipe and other obstacles force the clamp to rotate in space. Therefore, in order to accomplish this task properly, it is important that the haptic feedback restored to the user is realistic. The virtual engine mock-up is defined by 100, 000 triangles while the clamp is composed of 2, 000 triangles. The torque feedback device used in the previous experiment does not allow any translation. Thus, to be able to displace the clamp within the virtual environment, it is necessary to provide the mechanism with a translational DOF. For that purpose, a linear actuator designed and built at CEIT (Savall et al., 2008) is used. The displacement along this linear DOF is mapped into a displacement of the clamp along the axial direction of the pipe, and forces along this DOF are also displayed when collisions occur. Fig. 11 shows both the designed virtual environment and the haptic device used for the experiments. A virtual path from right to left along the route of extraction was performed. During this process, different types of collisions occurred between the clamp and the environment. Fig. 12 shows an example of the main possible stages during the extraction: 1. Initial position.  6. Final position. The clamp is disassembled. Fig. 12 shows the torque applied to the user in each axis (T r_x , T r_y and T r_z )a n dt h ef o r c e exerted by the linear actuator (F r_x ) computed by the collision response method. It also indicates the number of triangles in collision, in addition to the computational cost at each frame. Notice that in this figure data are shown from right to left according to the movement of the tool. Haptic feedback obtained gives a realistic perception of collision events and allows to perform the task properly. Unlike the previous scenario, designed to study the behavior of the method in situations with multiple collisions, in this case the aim is to simulate a real task. For this reason, although the complexity of the environment is higher, the number of simultaneous contacts decreases because the user corrects trajectory when a collision is detected. Fig. 13 is an augmentation of the third collision stage of Fig. 12 (frames 3450-3650) for torque feedback in the x axis, with and without applying the control algorithms. It can be seen that torque computed by the collision response is smooth and avoids abrupt changes. In addition, control algorithms improve the continuity of the feedback signal and apply it to the user at a sampling rate of 1 kHz. As in the previous example, in this case the number of transitions for the control algorithms is also 5.  0  1000  2000  3000  4000  5000  6000  7000  8000  9000  10000  11000   0  1000  2000  3000  4000  5000  6000  7000  8000  9000  10000  11000   0  1000  2000  3000  4000  5000  6000  7000  8000  9000  10000  11000   0  1000  2000  3000  4000  5000  6000  7000  8000  9000  10000  11000   0  1000  2000  3000  4000  5000  6000  7000  8000  9000  10000

Conclusions and future research
The real-time computation of the forces and torques in a virtual environment is a complicated task but a key point for the effectiveness of haptic systems. It is known that non-realistic or inappropriate haptic feedback has negative effects on the usability and leads to frustration when manipulating haptic systems. Therefore, it is very important to guarantee smooth and realistic haptic feedback. This chapter outlines a haptic rendering method that computes a proper haptic response in complex environments. It ensures improved feedback by seeking a compromise between continuity and computational cost. The method avoids abrupt changes in the haptic force direction and magnitude, thereby improving the overall stability of the haptic system. In order to validate the proposed method, two different scenarios containing complex collision examples, such as multiple contacts and geometrical discontinuities, have been used. The yielded results validate the applicability of the method in these types of interactions. In terms of future research, the authors plan to analyze the performance of the method from a perceptual perspective carrying out studies of human factors to improve the responsiveness. The authors also hope that the research included in this chapter will provide a better understanding of the many phenomena that challenge the development of improved haptic rendering methods able to display adequate force and torque feedback while preserving stability, and thereby improve performance of current haptic interfaces. There has been significant progress in haptic technologies but the incorporation of haptics into virtual environments is still in its infancy. A wide range of the new society's human activities including communication, education, art, entertainment, commerce and science would forever change if we learned how to capture, manipulate and reproduce haptic sensory stimuli that are nearly indistinguishable from reality. For the field to move forward, many commercial and technological barriers need to be overcome. By rendering how objects feel through haptic technology, we communicate information that might reflect a desire to speak a physicallybased language that has never been explored before. Due to constant improvement in haptics technology and increasing levels of research into and development of haptics-related algorithms, protocols and devices, there is a belief that haptics technology has a promising future.