N. Miyata, Statistical hand surface shape adjustment

Statistical surface shape adjustment for a posable human hand model N. MIYATA*†, D. NAKAMURA‡, Y. ENDO†, and Y. MAEDA‡ † Digital Human Research Center (AIST), Tokyo, JAPAN ‡Yokohama National University, Kanagawa, JAPAN

Abstract This paper proposes a method to reproduce a human hand surface shape precisely by statistically adjusting the shape by the skeletal subspace deformation (SSD) method. Plaster molds of the hand in variable postures were fabricated for one subject and their 3D shapes were obtained by computed tomography (CT) scanning. An SSD model was prepared for the subject to compute difference from the actual human hand. Landmarks were manually selected from each plaster mold model and from the SSD model. Each landmark position on a plaster mold model was compared with corresponding one on an SSD model posed to resemble the plaster mold model. The collected difference from all the plaster molds was analyzed with postures to estimate adjustment vector for landmarks on an SSD model in arbitrary posture. The rest of the vertices were corrected using adjustment vectors of the three nearest landmarks. Keywords: Hand Model, Skeletal Subspace Deformation.

1. Introduction To simulate plausible interaction between a hand model and a product CAD model for design assistance, it is required for the hand model to precisely reconstruct the human hand’s surface shape in arbitrary postures. However, deformation of the human hand is complex especially in the palm and the previous simple hand models (Magnenat-Thalmann et al. 1988; and other numerous extended works) had difficulty in the precise reconstruction of such complex deformation with validation. For better quality in the reconstructed shape, several researches took data-driven approaches that used a large number of shape data in various postures and realized a shape in a new posture by interpolation. Kry, et al. used results by finite element simulation to increase the amount of data, which were strictly not a living subject’s data (Kry et al. 2002). Kurihara, et al. used the hand shape data in five postures obtained through computed tomography (CT) scanning and interpolated them by a weighted pose space deformation method (Kurihara and Miyata 2004). However, the number of living subject’s hand capture through CT scanning cannot be increased to avoid harm to the subject’s health though it has an advantage in obtaining precise shape even with creases and its internal skeleton simultaneously. It also has no

*Corresponding author. Email: [email protected] 1

consideration of extending the method for different subjects. Rogers et al. collected a different subjects’ hand data in several postures (Rogers et al. 2008). It showed the possibility of modeling deformation according to the hand size. The data in this work were, however, limited in landmark positions and the resulted shapes were not smooth. From the viewpoint of light and smooth deformation, the skeletal subspace deformation (SSD) method (Magnenat-Thalmann et al. 1988) has been used widely. The model can be scaled to build a hand model of different size (Miyata et al., 2012) but the resultant shape often becomes thinner than the actual human hand shape because the method basically keeps the elevation of the original shape. Therefore, we propose a shape representation method that modifies an SSD model based on a statistical analysis of the shape difference from the measured actual human hand data in many different postures. To increase the number of precise shape data of the hand, plaster molds are fabricated. The derived statistical adjustment information is transferred to a homologous hand model of different size by scaling. After showing a concept of our method in section 2, details of the shape data collection are described in section 3 and those of modeling deformation of one subject in section 4. The proposed method’s

N. Miyata, Statistical hand surface shape adjustment

reproducibility is evaluated through the experiment in section 5 and is discussed in section 6.

Figure 1 A concept to model the deformation 2. Concept of a statistical shape adjustment For light and smooth deformation of the whole hand according to a posture, the SSD method is extended by complementing complex deformation characteristics of the actual human hand. Inside a square in light blue in Figure 1 shows our concept of building a deformation model for one base subject. To analyze the shape difference between an SSD model and the actual human hand, each measured hand shape is reconstructed by an SSD model. Though only one subject’s hand shapes were collected in this paper, it is important to build a deformation model which is comparable with that of the different subject’s. Therefore, we define a limited number of landmarks and analyze the shape difference to estimate necessary adjustment for the SSD method. Each of all the surface vertices but landmarks is adjusted using the estimated adjustment vectors for landmarks in the neighborhood. After modeling deformation for one base subject, the model will be applied to different subject. Given the dimensions of each part of the hand, an SSD hand model of any subject can be built by appropriately scaling a base SSD model (Miyata et al. 2012). Therefore, as shown in the bottom of Figure 1, an SSD hand model for a different subject is generated by scaling the base subject’s model and

2

Figure 2: 3D Plaster Mold Models in 23 postures the adjustment vectors will also be scaled using the same scaling factors. 3. Surface Deformation Measurement of a living subject’s hand To model surface deformation of an individual’s hand according to posture, we captured surface shape of one subject’s hand in various postures. A plaster mold for each posture was fabricated. Then three-dimensional shapes of them were obtained through computed tomography (CT) scanning. This process enabled us to avoid harm to health of the subject and to obtain precise hand shape on the palmar side without occlusion by curling fingers. Observed postures were decided considering the following four points. (1) A given posture could be recognized to be “different” from the rest of given posture conditions. (2) A subject could take the hand from an alginate cast without dissecting the cast around the fingers. (3) Included postures largely differ in adjacent metacarpophalangeal (MP) joints’ posture with each other. (4) Included postures distribute the hand joints’ range of motion as wide as possible.

N. Miyata, Statistical hand surface shape adjustment

The points (1) and (2) were considered because of fabricating plaster molds for surface shape observation. A subject should hold a requested posture for a while until alginate set enough to form a mold. But it is difficult to stably keep a same posture especially in the middle of the range of motion without looking at the hand directly. Consequently too small a difference of postures cannot be captured. Considering the above, we captured one subject’s hand in 23 postures. Figure 2 shows all the 3D models of plaster molds.

Figure 3: Landmarks defined on the hand (left) and picked on one of the 3D plaster models (right)

4. Deformation modeling for one subject 4.1. Posture reconstruction by an SSD model with landmark definition The plaster model in a straightened posture was converted into an SSD model. The skeleton of this SSD model was obtained using a MoCap system (Miyata et al., 2012). To reconstruct plaster mold postures by the SSD model, typical locations on the dorsal side were picked both on the SSD model and all the plaster mold models. These points were then used as done in reconstructing a posture from motion capture data. 162 landmarks were defined using creases that rather reflected a posture change than individual difference (Figure 3). Comparability of landmarks with different subjects was also considered in defining. Landmark positions were manually picked up from each plaster mold model and the SSD model in a straightened posture. We ignored a landmark that was fully occluded by surrounding skin and had no corresponding vertex of a model, which was often found on a crease when the hand joint was deeply flexed. Landmarks on an SSD model should be picked up appropriately for effective analysis of adjustment vectors in subsection 4.2. However it was difficult to pick up landmarks appropriately except for those on distinct creases. Such landmarks on an SSD model were, therefore, computationally adjusted so that each landmark on each reconstructed SSD model approached corresponding picked landmark on a plaster mold model appropriately over all the plaster mold models. 4.2. Difference modeling of landmark positions on a plaster model and an SSD model according to posture To generate surface shape for an arbitrary posture utilizing measured shapes in a limited number of postures, shape difference was analyzed with respect to the postures. Landmarks on the posed SSD model should coincide with corresponding one on a plaster hand model. So the difference at each landmark position was collected for all the plaster

3

a adjustment vector J

a landmark i J

pi

si

a plaster mold joint J’s coordinate frame an SSD hand mode

Figure 4 An adjustment vector for a landmark i

mold models and was analyzed as adjustment vectors. For each plaster mold model, an adjustment vector for landmark i (

J

s i )

was calculated in the joint

J’s coordinate frame using picked position on the plaster model ( model (

J

J

pi ) and that on the posed SSD

si ) as shown in equation (1) and Figure 4,

si  J pi  J si . J Collecting si from J

(1) all the plaster mold models

as a response variable, we conducted a multiple linear regression analysis of their x-, y-, and zcomponent respectively. Explanatory variables were some of the joint angles estimated to affect deformation at the landmark i’s position. A relationship between a landmark adjustment vector

J

si and a joint angle vector is then written

in equation (2), J

s i  βˆ i 

,

(2)

ˆ is a matrix composed of a regression where β i coefficient for each x-, y-, and z-component and

N. Miyata, Statistical hand surface shape adjustment

 

T

  1, θT . A vector θ is composed of some of the joint angles that were employed in regression analysis. 4.3. Adjustment vector in arbitrary posture Surface shape in an arbitrary posture is first formed by posing an SSD model as given posture. Then landmarks on it are adjusted using regression formulation obtained in subsection 4.2. Then other vertex k’s position

w

vk in world coordinates frame

is modified by adding adjustment factor

w

vk .

This vector is calculated as a weighted summation of the nearest three landmarks’ adjustment vectors as shown in equation (3),

vk  wl sl  wm sm  wn sn . (3) wl , wm , and wn are coefficients that correspond landmark l , m , and n , respectively. w

w

w

w

For all the vertices except for landmarks, corresponding three nearest landmarks are searched and weight coefficients to summate adjustment vectors were calculated beforehand. 5. Reproducibility Evaluation Experiment Plaster mold models were reconstructed to evaluate the reproducibility of the proposed statistical adjustment method. Because of employing statistical adjustment, shape error is inevitable even when reproducing surface shapes used in regression analysis. Figure 5 shows eight of the reproduced hand model surfaces. Left two columns contain SSD hand models without adjustment and right two columns contain those with the proposed adjustment. Color map was drawn based on the minimum distance for all the vertices on a SSD model from the nearest vertex on a plaster mold model. In each color map, blue corresponds to a small error and red a large error. The wrist in red in plaster mold 11 was because there was no corresponding part in the plaster mold model. Comparison of left and right color map in each row showed that the difference was reduced with the proposed adjustment methods. Minimum distances calculated when drawing a color map were averaged in each plaster mold and were summarized in Figure 6. It shows that the proposed adjustment reduces difference of a surface shape of the SSD model from the actual one in all the reconstructed postures. The relation between a region and shape error was examined by dividing the surface into 28 smaller regions as shown in Figure 7. As shown in Figure 8 that summarized the shape error for plaster mold model 5, the largest error occurred and was amended in the ball of the thumb (region 24). Figure 9 about shape error in region 24 showed that the averaged error was maximum in the plaster

4

mold model 5 as shown in Figure 6. The average of the error in region 24 was 13mm for the SSD model and was 3.6mm for the model with the proposed adjustment (73% improvement). Detail of the reconstructed shape for plaster mold model 5 in Figure 10 also showed that region 24 was largely amended by the proposed adjustment and resembled the actual subject’s hand well. 6. Discussion 6.1. Effect of defined landmark position In this paper, landmarks were defined mainly considering comparability to other subjects. For better reproducibility rather than comparability, however, landmarks can be defined considering the deformation characteristics. For example, each landmark in the palm which is not on creases is currently defined by equally dividing the connection line between landmarks on a crease as shown in Figure 3. Such landmark can be set, for example, where the curvature of the surface changes largely. 6.2. Reproducibility at landmark position Reproduced surface at each landmark position also had an error because of employing statistical adjustment. There are two types of error source. One is a mismatch of the employed linear model in regression analysis. The amount of deformation can change nonlinearly according to region. More complex models can be employed instead. The other is a bad correspondence between a picked landmark on a plaster mold model and that on an SSD model. In addition to careful manual picking up, refinement of the link model for an SSD model can reduce such error. 6.3. Adjustment method for all vertices but landmarks In the reproduced model with the proposed adjustment, the surface shape often looked flat triangle formed by the nearest landmarks. For example in Figure 10, the ball of the thumb is partially flat. This was due to the method to calculate an adjustment vector for all vertices but landmarks. Such vertices are currently adjusted by a weighted summation of the adjustment vectors of the three nearest landmarks where the weight coefficients were determined according to the ratio of interior division. When the three adjustment vectors are parallel, length of the summated vector is larger than the shortest one and elevation of the surrounded area is preserved. However, when the adjustment vectors directs largely different from each other, the length of the summated vector always becomes shorter than any of the landmarks’ adjustment vectors and the shortest at the centroid of those landmarks. In highly elevated area, adjustment vectors of the related landmarks often

N. Miyata, Statistical hand surface shape adjustment

directs largely different. Such effect becomes indistinctive by smoothing operation to the whole shape. However, this should be improved for better reproducibility.

Figure 7 Regions for error evaluation 14.0 SSD Mean Error [mm]

12.0

SSD+Adjustment (Proposed)

10.0 8.0 6.0 4.0 2.0 0.0 all 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627 Color Area ID

Figure 8 Reproduction errors in plaster posture 5 according to region 14.0

Figure 5 Some of the reproduced plaster molds by the SSD (left two columns) and the proposed method (right two columns) both with error from a corresponding plaster mold model drawn as a color map.

Mean Error [mm]

12.0

SSD

SSD+Adjustment (Proposed)

10.0 8.0 6.0 4.0 2.0 0.0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Plaster Posture ID

Figure 9 Reproduction errors in region 24 of plaster mold 5

7. Conclusion

Figure 6 Averaged difference in all the plaster molds' Posture

5

This paper proposed the method to realize the precise shape of the hand according to a posture by adjusting a simple SSD model’s surface based on statistical analysis of the captured surface shapes of a living subject in various postures. One male subject’s hand shape was collected in 23 postures by fabricating plaster molds and scanning them by CT. 162 landmarks were defined on the palmar side and the shape difference at landmarks between

N. Miyata, Statistical hand surface shape adjustment

similarly posed SSD model and plaster mold model were analyzed with a posture. Derived linear regression formulation was used to estimate adjustment vector for each landmark in arbitrary posture and other vertices but landmark were adjusted by interpolating adjustment vectors for the nearest three landmarks. Reproducibility of the proposed adjustment method was validated by reproducing the hand in the postures for which the plaster mold models were fabricated. Comparison of the reproduced shape with that of the plaster mold model showed that the SSD model’s shape was refined by 73 % and resembled the actual hand’s shape well in the ball of the thumb. Our future works include refinement of landmarks’ location, regression model for adjustment vector estimation, and adjustment method of all vertices but landmarks. Now we are testing the applicability of the deformation model for one base subject to different homologous hand shape assuming that the deformation characteristics between them are similar. References N. Magnenat-Thalmann, et al., 1988. Jointdependent local deformations for hand animation and object grasping. Graphics Interface 88, 26-33. M. S. Rogers, et al., 2008. A three-dimensional anthropometric solid model of the hand based on landmark measurement. Ergonomics, Vol. 51, No. 4, 511-526. P. G. Kry, D. L. James, and D. K. Pai, 2002. EigenSkin: Real Time Large Deformation Character Skinning in Hardware. Proceedings of ACM SIGGRAPH Symposium, 153-160. T. Kurihara and N. Miyata, 2004. Modeling deformable human hands from medical images, Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation, 357-365. N. Miyata, et al., 2012. Individual Hand Model to Reconstruct Behavior from Motion Capture Data. Int. J. Human Factors and Modelling and Simulation, Vol. 3, No. 2, 147–168.

6

Plaster

SSD

Proposed

Figure 10 Reproduction of plaster mold 5. An SSD model without adjustment (upper middle) and with adjustment were respectively compared with the plaster mold model (upper left) as shown in the color map (lower row).

Statistical surface shape adjustment for a posable human hand model

statistical analysis of the shape difference from the measured actual human hand data in many different postures. To increase the number of precise shape.

972KB Sizes 3 Downloads 226 Views

Recommend Documents

Statistical surface shape adjustment for a posable human hand model
statistical analysis of the shape difference from the measured actual human hand data in many different postures. To increase the number of precise shape.

Regularized Active Shape Model for Shape Alignment
locating a landmark. ai equals to 0 implies that classifier can perfectly predict shape's boundary; while ai equals to 1 means classifier fails to ..... asm/index.html.

Regularized Active Shape Model for Shape Alignment
PCA is still unsmooth when the shape has a large variation compared with the mean ...... Sparse Data. 3D. Data Processing, Visualization and Transmission, 2004.

cued speech hand shape recognition
paper: we apply the decision making method, which is theoretically .... The distance thresholds are derived from a basic training phase whose .... As an illustration of all these concepts, let us consider a .... obtained from Cued Speech videos.

A Statistical Model for Estimating Probability of Crack ...
Index Terms—Detection, Inspection, Health monitoring, ... Alexandra Coppe is Graduate Research Assistant with University of ... France (email: [email protected]).

The Surface Station Model and Surface Charts
1. The Surface Station Model and Surface Charts. Surface Weather. Observation. Station Model. A. B. C. D. G. F. E. H. I. J. • A = Total Cloud Cover. A. B. C. D. G. F. E. H.

Deep Learning Guided Partitioned Shape Model for ... - IEEE Xplore
Abstract—Analysis of cranial nerve systems, such as the ante- rior visual pathway (AVP), from MRI sequences is challenging due to their thin long architecture, structural variations along the path, and low contrast with adjacent anatomic structures

Weighted Partitioned Active Shape Model for Optic ...
the best nearby point based on the appearance cost function, we have .... degrees of freedom in shape training and requires less training examples to build an ... Computer vision and image understanding 61, 38Y59 (1995). 3. Lamec er, H., et ...

Fan Shape Model for Object Detection
This flexibility allows FSM to tolerate large ... Scale variance is big challenge for object detection. ... To summarize, there are five main advantages of the fan ... age data. Previous fundamental research of the part-based object model can be foun

Shape Optimization for Human-Centric Products with ...
Feb 27, 2014 - Email: [email protected]. Abstract. In this paper .... (Left) A given design is defined on the template human body model H. (Middle) The ...

Hemicraniectomy: A New Model for Human ...
Berger improved his EEG recording technology, he was able to acquire EEG from .... approximate underlying Brodmannʼs area (BA; Koessler et al., 2009; Okamoto ...... (Canolty et al., 2006) that information obtained through interfrequency ...

Hemicraniectomy: A New Model for Human ...
rhythms, we hypothesized that that data from the elec- trodes placed over the ... patient with at least 4 weeks recovery time following their ... was 3 in the field.

The shape of human gene family phylogenies
have erased any trace of this event from many of our gene families, particularly if massive gene loss quickly followed the polyploidy events [35]. Similarly, it is not ...

Hand Gesture Recognition for Human-Machine ...
achieve a 90% recognition average rate and is suitable for real-time applications. Keywords ... computers through hand postures, being the system adaptable to ...

The shape of human gene family phylogenies
form new lineages (the actual rate of splitting has no effect). This is often ..... notebook for calculating expected values of this index under the ERM and PDA ...

Measuring-Slipperiness-Human-Locomotion-And-Surface-Factors.pdf
Retrying... Whoops! There was a problem previewing this document. Retrying... Download. Connect more apps... Try one of the apps below to open or edit this item. Measuring-Slipperiness-Human-Locomotion-And-Surface-Factors.pdf. Measuring-Slipperiness-

Statistical Model Checking for Cyber-Physical Systems
The autopilot is a software which provides inputs to the aircraft's engines and flight control surfaces (e.g., ..... Therefore, instead of try- ing to come up with the optimal density, it may be preferable to search in a ..... optimization. Methodolo

Statistical Model Checking for Markov Decision ...
Programming [18] works in a setting similar to PMC. It also uses simulation for ..... we use the same input language as PRISM, many off-the-shelf models and case ... http://www.prismmodelchecker.org/casestudies/index.php. L resulting in the ...

Uniform Multilingual Multi-Speaker Acoustic Model for Statistical ...
single-speaker system from small amounts of data for that lan- ... In recent years, statistical parametric speech synthesis has seen .... analysis components.