Human-like Walking with Straightened Knees, Toe-off and Heel-strike for the Humanoid Robot iCub Zhibin Li*, Bram Vanderborght**, Nikos G. Tsagarakis*, Darwin G. Caldwell* *Italian Institute of Technology, Genova, 16163 Italy (Tel: 39-010-71781481; e-mail: [email protected]). **Vrije Universiteit Brussel, Brussels, 1050, Belgium (e-mail: [email protected]) Abstract: Most humanoid robots don’t walk in a very human-like manner. Their bent knees ensure a constant centre of mass height but this requires high motor torques. Also, humanoids walking without toe-off and heel-strike phases are considered to be considerably less energy efficient. Therefore, it is desirable to design a walking trajectory generator that has both straightened knees and toe-off and healstrike phases. In this paper, we address this issue by designing pattern generators for varying both the hip height and the feet trajectories (toe-off and heel-strike phases). The dynamic simulation of a child robot “iCub” demonstrates successful gaits in a human-like manner. The knee joint torque required by the proposed strategy is reduced, compared to conventional bent knee walking. Additionally, the simulation also reveals a redistribution of joint torques among the hip, knee and ankle in the human-like walking, which is more energy efficient. Keywords: Human-like Walking, Straightened Knee, Toe-off, Heel-strike.

1. INTRODUCTION Humanoid robots can now achieve good performance across various walking gaits [Matsui, et al., 2005], [Ill-Woo, et al., 2006], [Ogura, et al., 2004]. However, this has only been achieved through the development of sophisticated control schemes which deal with the issues of balance, stability, and the Centre of Mass (COM) coupling in the sagittal and frontal planes. This has been addressed using a variety of techniques. The overall control problem was resolved by constraining a constant COM height, to simplify the non-linear spatial motion of the COM into a linear form. Therefore, modern control theory can be used to control bipedal walking, which is a multi-dimensional target. However, this in turn led to the knee singularities occurring at certain times. These singularities were typically addressed by walking with bent knees which kept a constant COM height and prevented singularities. Practically, however, a real robot’s actuators have limited power and thus can exert only a limited torque and velocity. Bent knee walking, especially, demands large knee torques and there would be considerable hardware design (and aesthetic) benefits if a stable gait could be found which allows more up-right walking. In humanoid walking, the Zero Moment Point (ZMP) criterion [Vukobratovic and Borovac, 2004] is widely used for maintaining stability. Assuming that the robot is represented by a single mass model, the ZMP is the point where the resultant force of inertia force and gravity penetrates the ground. In the dynamic walking, the robot is considered to be stable when ZMP stays in the foot support polygon. Therefore, to enhance stability, researchers generally use flat feet trajectory to keep a maximum footground contact, and keep the ZMP trajectory at the centre of

the foot support polygon, resulting in a largest safety region. This produces the classical robot gait which has an awkward motion when compared to human walking. Therefore, although conventional humanoid walking with bent knees and flat feet can guarantee the stability of the robot, the gait is still unnatural and energetically inefficient. These limitations have inspired new research into pattern generation to create a more human-like walking gait. The work in [Ogura, et al., 2004] achieved a more humanlike walking gait for the WABIAN-2 robot using stretched (less bent) knees while avoiding singularities. In this robot, a pattern generator consisting of two sinusoid motions is used to predefine the knee joint trajectories. An additional and perhaps even more important benefit of this form of motion is that stretched knees require less torque and consume less energy than conventional walking with bent knees. Handharu, et al. [Handharu, et al., 2008] found a hip trajectory satisfying ZMP, and then solve the inverse kinematics by defining an initial foot trajectory. They then modified the initial knee trajectories to obtain the stretch motion by cubic interpolation. Accordingly, the feet motion is recalculated to find the inverse kinematic solution for the new knee joint trajectories. Both of these methods plan the knee motions in joint space based on the proposed, predetermined strategies. In this paper, a pattern generation method is presented for a human-like walking gait with increased straightening (but not fully straightened) of the knees. This is supplemented with toe-off and heel-strike phases, which is formulated in Cartesian space without predefining or redesigning knee trajectories in the joint space. Since the walking pattern executes a spatial motion, parameter tuning becomes more

straightforward and intuitive in Cartesian space to produce various walking gaits. To achieve a walking behaviour that is more comparable with humans, the constant height constraint in the commonly used cart-table model [Kajita, et al., 2003] is relaxed which creates an undulating vertical hip motion, and foot trajectory generator is designed to enable toe-off and heel-strike motion. This study shows that with these modifications singularities are still avoidable through the correct design of the hip and feet trajectories. Three parameters are proposed to tune the gait pattern for singularity avoidance. This paper is organized as follows. In section 2, the cause of the singularity issue is analyzed and solutions are proposed. Section 3 describes the trajectory generation of both the hip height and the feet motion including toe-off and heel-strike, and the overall control scheme. Section 4 validates the effectiveness in the dynamic simulation, and compares the knee joint angles, waist motions, and joint torques with that required by conventional bent knee walking. 2. CONSTRAINT OF THE CART-TABLE MODEL 2.1 Cart-table Model

especially at the beginning and the end of the double support phase. Although this solves the singularity issue, when the robot hip passes over the ankle joint in the mid-stance, the knee joint angle becomes large, resulting in high motor torque to counteract gravity, as illustrated in Fig. 2 (b).

Fig. 2. Kinematic analysis of singularity issue. However, with the toe-off and heel-strike motion, the kinematic workspace of the legs is able to cover the footholds even with a higher hip position. Fig. 2 (c) shows how the toeheel motion prevents the knee singularities when the hip position is higher. More beneficially, as demonstrated in the Fig. 2 (d), a higher hip position in the single support phase decreases the knee joint angles, which results in lower torques. 3. TRAJECTORY GENERATION

(a)

(b)

In this study, the spatial COM motion is calculated horizontally and vertically. A preview control is applied to generate a horizontal motion, and a hip trajectory generator produces a smooth vertical motion. This section analyzes the feasibility of integrating the hip trajectory generator.

Fig. 1. Cart-table model (a) and bent knee walking (b).

3.1 Variation of Height as a Parametric Disturbance

The cart-table model proposed by Kajita, et al. [Kajita, et al., 2003] assumes a simplified robot model where a running cart of mass m is placed on a pedestal massless table, as shown in Fig. 1 (a). If the cart, which represents the robot’s COM, rests outside the foot area of the table (supporting foot polygon), the robot will tip over. However, the ZMP can be positioned inside the support region by choosing a proper acceleration, and hence maintain stability. A preview controller is used to generate the COM’s desired horizontal motion given various footholds. Also, the conventional scheme requires the feet to be parallel to the ground.

The preview control is based on the cart-table model, while the hip trajectory generator violates this constant height constraint. However, Stasse et al. [Stasse, et al., 2009] have previously shown that the initial assumption of the cart table model can be slightly relaxed. The hip height variation is allowed since the second preview loop, acting as a dynamic filter, deals with the disturbances caused by this vertical hip motion. This study allows us to introduce the proposed hip trajectory generator to create a varying hip height motion.

This conventional scheme is realized in the OpenHRP3 simulator [Kanehiro, et al., 2004] based on the preview control with a constant hip height. In Fig. 1 (b), the time elapsed snapshots show the legs of a walking robot. The knee joint in mid-stance is bent due to the constant hip height constraint. 2.2 Relaxing the Hip Height Constraint It is therefore desirable to remove the constant hip height constraint. However, Fig. 2 (a) shows that an increase in the hip height may cause a singularity in the double support phase with flat feet. In the cart-table model, the settled height is reduced such that no singularity occurs at the knee joints,

Letting x and z denote the 2D COM’s position, and be the gravitational constant, then the ZMP equation of a single mass model is: (1) The simplified ZMP equation used in cart-table model is: (2) The ZMP error caused by this difference is: (3) Consider an acceleration variation [-0.25, 0.25] / , a height variation [0.41, 0.44]m and an average horizontal

acceleration 1 / , a parameter scan shows that the ZMP error created by the vertical movement is negligible, as shown in Fig. 3. Therefore, if a designed hip pattern has both the height variation ∆ and acceleration variation ∆ bounded in a small range, the ZMP error is minor. -3

x 10 2

ex: m

1

0

For instance, a gait cycle of 1 , =0.02m only results in maximum acceleration of 0.25 / . Therefore, the hip pattern generator guarantees: 1) a smooth hip position trajectory which reaches its peak in mid stance phase and its trough in mid double support phase; and 2) a continuous vertical acceleration profile without jerks as in (5), which can be easily bounded by setting one parameter .

-1

3.3 Foot Trajectory Generation

-2

-3 0.44 0.435 0.2

0.43 0.1

0.425 0

0.42 -0.1

0.415 0.41

z

-0.2

zdd

Fig. 3. ZMP error created by the vertical movement. 3.2 Hip Pattern Generator As previously discussed, the vertical acceleration, , mainly dominates the errors due to the varying hip height. This section considers a hip pattern generator that produces a smooth hip trajectory with a small acceleration variation ∆ .

Fig. 4. Parameters of the hip pattern generator. As shown in Fig. 4, the pattern generator defines a sinusoidal hip motion by correlating hip position in and axes by: 0.5

2

0.5

(4)

Fig. 5. Toe-off and heel-strike foot trajectory. As shown in Fig. 5, the foot trajectory has four phases, namely stance phase, , toe-off phase, , heel-strike , and swing phase, . A state machine is phase, applied to switch between these four states, and a 5th order polynomial is used to interpolate those separate trajectories smoothly in Cartesian space. In order to avoid “stabbing motions” at toe-off or heel-strike, both the boundary angular speed and acceleration are set to zero. The method developed in this paper is similar to the work in [Huang, et al., 2001]. and are the maximum angle at toe-off and heelstrike, respectively, and the corresponding position is denoted , , and , . is by , is the ankle position of the previous stance foot and position of the new stance foot. The · is the displacement from when foot swings at the highest · , , where 0 0.5. position 3.4 ZMP Trajectory Generation

where and are the ankle position of front and rear foot, respectively, is the initial hip height, and is the amplitude of sinusoidal pattern. A larger z produces straight knees in the single support phase. The larger A gives a lower hip position in the double support, so it is more likely to avoid knee singularities occurring. The two parameters z and A can be tuned to generate various hip patterns. It should be noted that we shall not use sinusoidal equation to vary the hip height as a function of time, because this results in a distorted sinusoidal pattern in Cartesian space, as the horizontal velocity x is not constant. In contrast, our method as a function of as in (4). Substituting the defines step length = into (4). The amplitude of the acceleration caused by the vertical motion is: sin 2

(5)

Fig. 6. ZMP trajectory planning.

In order to generate the ZMP trajectories needed for the toeoff and heel-strike feet, the ZMP trajectory is evolved smoothly from the rear to the front foot. In Fig. 6, the red dots represent the start and the end point of ZMP transfer in different walking phases. In single support, the ZMP transfers from the back to the front of the stance foot with safety margins, shown in Fig. 6 (a). Figs. 6 (b), (c), and (d) are the double support phase including the heel-strike and toe-off motion. Due to the heel and toe motion, the support polygon changes which is highlighted by the yellow area. In our study, safety margins are always guaranteed no matter how the support polygon varies. Meanwhile, no under-actuated foot rotation is exploited since it is unstable for the robot. Instead, when one foot rotates, another foot always maintains the ground contact, as shown in Figs. 6 (b), (c), and (d). It can be seen that the ZMP and foot trajectory are closely interlinked, because the ground contact is determined by the feet trajectories. The timing of the ZMP transfer is crucial. For example, in Fig. 6 (b), from stance to pre-heel-strike, the ZMP starts to move outside of the stance foot after the heelstrike occurs. Another case is from post-toe-off to next stance. In this instance, the ZMP should move inside the new stance foot before the rear foot leaves the ground, as shown in Fig. 6 (d).

within a certain range, so errors can be compensated for by the second preview control loop. The parameter tuning is simple and intuitive according to its kinematic meanings. The standard procedure is proposed to avoid singularity and obtain desired gaits. regulating the knee joint 1) Set a constant hip height angle in mid stance phase (the height where the hip passes over the ankle). Usually this is close to the full leg length. 2) Increase the parameter to enlarge the amplitude of vertical variation, to avoid knee singularities in the double support phase. and heel-strike angle , 3) Increase the toe-off angle and at the same time, reduce parameter (by setting · , we can simplify tuning by lumping together). Therefore, the toe-off and heel-strike and posture compensates for the increase of hip height when decreasing , to prevent the knee singularities. 4) Increase if knee joint is not straightened sufficiently; when the knee joint is too close to the or reduce singularity and its angular velocity is too high. Repeat the above procedure until a desired parameter set is found.

3.5 Overall Control Scheme 4. SIMULATION OF HUMAN-LIKE WALKING

xcom, ycom Objective Locomotion Parameters Preview Control

Foothold Planning

u

_ Foot Trajectory

ZMP Trajectory

p

p ref

Inverse Kinematics

xhip, yhip

COM to Hip position approximation

xhip

+

pref

Calculation of Multi-Body ZMP

-

4.1 Robot Model

Delta xcom, Delta ycom

Preview Control

p'ref

Hip Pattern Generator

p

ZMP Error

xfr, yfr, zfr xfr, yfr, zfr

xhip yhip

zhip

Human-like walking is classified into three categories according to the step length: (1) small gait which the step length is smaller than 30% of the full leg length; (2) medium gait when the step length is between 30% and 50% of the full leg length; (3) large gait when the step length is larger than 50% of the full leg length. This study investigates small and medium gaits only but avoids the large gait due to the decrease in stability.

+

-

u

p'

Delta xhip COM to Hip Delta yhip position + approximation

+

x'hip, y'hip zhip

(a) Inverse Kinematics

(b)

Joint Trajectories

Fig. 7. Overall control scheme. The pattern generators for the hip and foot trajectory are incorporated in the preview control scheme based on a carttable model, shown in Fig. 7. The compatibility is based on the premises that the errors yielded by and are constrained

Fig. 8. iCub robot (a) and its model (b) in the simulation. We use the proposed scheme for the control of a 3.5 years old child humanoid robot iCub, as shown in Fig. 8 (a). The robot model is built in the dynamic simulator OpenHRP according to its physical parameters [Tsagarakis, et al., 2007].

The dynamic simulations were implemented in OpenHRP3. Joint trajectories of the conventional robotic and the humanlike walking are generated for small (0.15m) and medium steps (0.2m), respectively.

Knee Tracking: degree

4.2 Conventional and Human-like Walking

Step Length 0.15 m 60

Straightened Knee Bent Knee

50 40 30 20 10 0

0

2

4

6

8

10

12

14

16

18

20

Time: s

Constant Height

Constant Height

Flat feet

Flat feet

(a)

(b)

Knee Tracking: degree

Step Length 0.2 m 60

Straightened Knee Bent Knee

50 40 30 20 10 0

0

2

4

6

8

10

12

14

16

18

20

Time: s

Fig. 10. Comparison of knee joint angles. 0.44

Conventional 0.15m Human-like 0.15m

0.435

0.43

Heel-strike

Variable Height

Variable Height

Heel-strike

0.425

0.42

Toe-off

Toe-off

0.415 0.3

0.35

0.4

0.45

0.5

0.55

0.6

Waist X position: m

Fig. 9. Conventional and human-like walking (step length 0.15m and step length 0.2m). In the dynamic simulation, the robot successfully performs stable gaits as shown in Fig. 9. Figs. 9 (a) and (b) show conventional walking with step lengths of 0.15 and 0.2 , with bent knees and flat-feet, which is commonly observed in humanoids walking. Figs. 9 (c) and (d) show human-like waking with step lengths of 0.15 and 0.2 . It can be seen that the robot places its front foot with a heel-strike and ends its double support with a toe-off. The human-like walking is characterised by its toe-off and heel-strike which is absent in the conventional robot motion, as illustrated in Fig. 9. The constant hip height feature is marked by the red straight line in Figs. 9 (a) and (b), while the variable hip height is highlighted by a wavy red curve in Figs. 9 (c) and (d). 4.3 Analysis of Dynamic Simulation

0.425 0.42 0.415

0.3

0.4

0.45

0.5

0.55

0.6

0.65

0.7

Fig. 11. Waist position data obtained from simulation. Figs. 12 and 13 show the joint torques of hip pitch, knee, and ankle pitch for step lengths of 0.15 and 0.2 respectively. In the human-like walking, knee joint torques has an obvious decrease at the mid-stance phase. Step Length 0.15 m

100

Knee Torque: Nm Ankle Torque: Nm

In Fig. 11, the waist motion from the simulation displays a sinusoidal pattern in human-like walking, which matches with our pattern design. By contrast, in conventional walking, the hip height is almost constant. The simulation shows that with the proposed hip and feet pattern generation, the hip height can be generally raised higher than that in conventional walking, so the knee joint torques are reduced.

0.35

Waist X position: m

Conventional Human-like

50 0 -50 -100 -150

Fig. 10 shows the knee joint tracking data from the dynamic simulation. In the mid stance phase, the knee joint angles are 12° (step length 0.15 ) and 14° (step length 0.2 ) in humanlike walking, while for conventional walking they are 29° (step length 0.15 ) and 36° (step length 0.2 ).

Conventional 0.2m Human-like 0.2m

0.43

0.41

Hip Torque: Nm

(d)

Waist Z position: m

0.435

(c)

0

2

4

6

8

10

12

14

16

18

100 50

Mid-stance

0

Time: s 20

Conventional Human-like

-50 -100 -150

0

2

4

6

8

10

12

14

16

18 Time: s20

100

Conventional Human-like

50 0 -50 0

2

4

6

8

10

12

14

16

18

20

Time: s

Fig. 12. Torques of the sagittal joints (step length 0.15m).

Step Length 0.2 m

Hip Torque: Nm

150

50 0 -50 -100 -150

Knee Torque: Nm

Conventional Human-like

100

0

4

6

8

10

12

14

16

18

20

Time: s Conventional Human-like

Mid-stance 0

-100 0

Ankle Torque: Nm

2

100

Moreover, the dynamic simulation reveals an interesting redistribution of the joint torques. In the bent knee walking, the knee bears the highest torque. Meanwhile, the torque of each joint is very different. In contrast, in human-like walking, the joint torques are more equally distributed across the hip, knee, and ankle, instead of concentrating on a single joint. The standard deviation of sagittal joint torques is listed in Table 1, which indicates this apparent phenomenon of torque redistribution.

2

4

6

8

10

12

14

16

18

20

ACKNOWLEDGMENTS

Time: s

150

This work is supported by the FP7 European project AMARSi (ICT-248311).

Conventional Human-like

100 50 0

REFERENCES

-50 -100

0

2

4

6

8

10

12

14

16

18

20

Time: s

Fig. 13. Torques of the sagittal joints (step length 0.2m).

Trms  : Nm

45 40 35 30 25 20 15 10 5 0

38.9 Conventional 34.2

32.4 30.5

Human‐like 30.6 26.9

25.4 19.4

23.6

21.0 16.316.6

Hip Pitch Knee Ankle PitchHip Pitch Knee Ankle Pitch (0.15m) (0.15m) (0.15m) (0.2m) (0.2m) (0.2m)

Fig. 14. Root mean square of the joint torques. Fig. 14 compares the root mean square (RMS) of the joint torques for comparing energy efficiency. For a step length of reduces 25.75% (from 34.2 to 25.4Nm). 0.15m, the knee decreases 21.34% For a step length of 0.2m, the knee (from 38.9 to 30.6 Nm). However, in both cases, there are increases in hip and ankle joint torques. The increase in ankle joint is due to the toe-off and heel-strike trajectory, since the actuators consume slightly more energy to execute such motion. Because of the sinusoidal hip joint motion, the hip actuators need to be more active to maintain the body in an upright posture. It should be noted that the upper body has large mass and inertia, so it requires more hip torques. In spite of these increases, the energy consumption of humanlike walking is still less than the conventional bent knee walking. The total RMS torques of the sagittal joints is 69.9 Nm (conventional) and 63.0Nm (human-like), and 93.1Nm (conventional) and 90.0Nm (human-like) for step lengths of 0.15m and 0.2m, respectively. Table 1. Standard deviation of joint torques (Nm) Step Length (cm) 15 20

Conventional 9.5588 7.6851

Hunan-like 4.3947 2.8208

Huang, Q., Yokoi, K., Kajita, S., Kaneko, K., Arai, H., Koyachi, N., and Tanie, K. (2001). Planning walking patterns for a biped robot. IEEE Transactions on Robotics and Automation, 17 (3), 280–289. Handharu, N., Yoon, J., and Kim, G. (2008). Gait pattern generation with knee stretch motion for biped robot using toe and heel joints. International Conference on Humanoid Robots, Daejeon, Korea, 265 – 270. Ill-Woo, P., Jung-Yup, K., Jungho, L., and Jun-Ho, O. (2006). Online free walking trajectory generation for biped humanoid robot KHR-3(HUBO). IEEE International Conference on Robotics and Automation, 1231–1236. Kanehiro, F., Hirukawa, H., and Kajita, S. (2004). OpenHRP: Open architecture humanoid robotics platform. The International Journal of Robotics Research, 23 (2), 155– 165. Kajita, S., Kanehiro, F., Kaneko, K., Fujiwara, K., Harada, K., Yokoi, K., and Hirukawa, H. (2003). Biped walking pattern generation by using preview control of zeromoment point. IEEE International Conference on Robotics and Automation, 2, 1620–1626. Matsui, T., Hirukawa, H., Ishikawa, Y., Yamasaki, N., Kagami, S., Kanehiro, F., Saito, H., and Inamura, T. (2005). Distributed real-time processing for humanoid robots. IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, 205–210. Ogura, Y., Lim, H.O., and Takanishi, A. (2003). Stretch walking pattern generation for a biped humanoid robot. IEEE/RSJ International Conference on Intelligent Robots and Systems, 1, 352–357. Stasse, O., Verrelst, B., Vanderborght, B., and Yokoi, K. (2009). Strategies for humanoid robots to dynamically walk over large obstacles. IEEE Transactions on Robotics, 99, 1–8. Tsagarakis, N., Metta, G., Sandini, G., Vernon, D., Beira, R., Becchi, F., Righetti, L., Santos-Victor, J., Ijspeert, A., and Carrozza, M. (2007). iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research. Advanced Robotics, 21 (10), 1151–1175. Vukobratovic, M. and Borovac, B. (2004). Zero-Moment Point - Thirty Five Years of its Life. International Journal of Humanoid Robotics, 1, 157-173.