Hybrid Architecture for Kick Motion of Small-sized Humanoid Robot, HanSaRam-VI Jeong-ki Yoo, Yong-Duk Kim, Bum-Joo Lee , In-Won Park, Naveen Suresh Kuppuswamy and Jong-Hwan Kim Robot Intelligence Technology Laboratory, Dept. of EECS, KAIST, Guseong-dong, Yuseong-gu, Daejeon, 305-701, Republic of Korea {jkyoo, ydkim, bjlee, iwpark, naveen, johkim}@rit.kaist.ac.kr

ABSTRACT This paper presents a hybrid architecture designed for the kick motion of a humanoid robot. The principal components of this architecture are vision system, path planner and motion generator. The vision system utilizes captured images for calculating ball and goal positions and simple self-localization using a proposed image preprocessing technique. Based on this information, appropriate path and motion procedures are calculated in the path planner and motion generator embedded within the PDA. As all deliberative decision is made in above three modules, there is temporal independence between deliberative layer and reactive layer of robot control. By virtue of this, all procedures of behavior selection and execution could be done efficiently according to the situation. The performance of the proposed scheme are demonstrated through the penalty kick of FIRA HuroSot using small-sized humanoid robot, HSR-VI, developed by RIT lab in KAIST.

1. INTRODUCTION Research in the development of Humanoids represent the cutting edge in Robotics. Honda ASIMO [1], WABIAN of Waseda University [2], HRP [3], and HanSaRam [4] stand testimonial to the rapid progress and development in this area. Current research on generating walking pattern generation is mainly based on the inverted pendulum model [4]. From the predefined model, several stability maintenance algorithms such as impedance control [5], online (realtime) balance control during walking [6] and time domain passivity control of the landing impact force have been proposed [7]. For stable walking, force control and walking pattern planning algorithms were developed, and their efficiency was shown [8],[9]. Apart from the research topics related to walking algorithms, path planning and vision processing also remain important research issues. For a wheeled robot,

many solutions on localization and object detection are developed, including those with omni-vision camera [10]. However, for a humanoid robot, sensor capabilities are limited, in order to mimic the human way of sensing. In most cases, humanoid robot uses stereo vision to obtain depth information from captured images [11]. By using the result, appropriate paths are generated according to landmarks or heuristics. This paper describes the recent progress and development in humanoid robot, HSR-VI, especially focusing on its hybrid architecture. The derivative layer including the vision module, situation detector, path planner and motion planner is implemented in a PDA mounted on the head of a small-sized humanoid robot. The walking pattern generation is completely performed in the embedded computer, which is installed on the back of HSR-VI. Therefore the deliberative layer operations could be performed independently on the executive layer. In vision processing, edge and color based detection algorithms are combined for the performance of detecting ground area. Using the result, landmark based localization is performed and behavior selection and scheduling are also performed according to the situation. The validity of the proposed hybrid architecture is confirmed through penalty kick experiments of HSR-VI developed in RIT Lab., KAIST in 2004. The reminder of this paper is organized as follows: Section 2 shows the HSR-VI. Section 3 explains the whole scheme of hybrid architecture. Section 4 presents the vision system implemented in deliberative layer. Section 5 shows the experiments, the penalty kick, through the snap shot of movie clip. Finally, concluding remarks follow in Section 6.

2. HanSaRam-VI HSR-VI developed in 2004, as shown in figure 1, had 25 DOFs and consisted of 12 DC motors for a lower body and 13 RC servo motors for a upper body. Its height and weight were 52 cm and 4.6 kg, respectively. The design of

its lower body was focused on the delivery of sufficient torque and zero backlash with DC motors and harmonic drives. The main difference of HSR-VI compared with past versions of HanSaRam series was in the design of lower body. It was simplified by designing the harmonic drive and DC motor as a single module. Its walking motion was generated on-line through three-dimensional linear inverted pendulum mode [3]. Since it effectively represented the whole dynamics of humanoid by the inverted pendulum, walking pattern could be generated online. Moreover, turn and stop motions could be easily generated. Since the width between two z-axis of pelvis was designed to be narrow, it could walk properly with less shakes on hip compared to the previous HSRs' walking. Moreover, initial positioning of its posture was automatically setup by using a photo interrupter and a revolving disk for the DC motor control. RTLinux was used for the control of HSRVI, and four FSRs per each sole of foot were used to measure the ZMP. HSR-VI had the ability for fully independent sensing, processing, and locomotion.

In order to follow the path, appropriate motion procedure is generated by the motion planner, which is also implemented within the PDA. Through the predefined motion set and its elapsing time per each motion, deliberative layer sends proper motion codes to executive layer one by one. Recognition of the current situation, decision on the next motion set, and process of the images are all performed in the deliberate layer like how humans do so in their head. Embedded Computer

PDA with CAMERA Deliberative Layer

Executive Layer

Vision Module RS 232 Motion Generator

Ball / Goal Detection

Localization

Walking pattern generator Aperiodic Motion generator

Motion Num Done Signal

Sel

Reactive Layer

Situation Detector

Posture control

Motors

Motion Planner FSR0

FSR1

FSR2

…

Path Planner

FSR 7

Figure 2: Hybrid control architecture for HSR-VI

4. VISION SYSTEM

4.1. System structure

(a) HSR-VI

(b) HSR-VI with PDA head

Figure 1: HanSaRam-VI

3. HYBRID ARCHITECTURE The control architecture of HSR-VI in figure 2 is composed of two components; one is an embedded computer which generates appropriate walking patterns and the other is a PDA, having CMOS camera, which captures and processes image for localization and object detection. As the two components of the architecture are implemented independently, walking pattern generation and perception of situation could be performed in parallel. The situation detector, which is implemented in the deliberative layer in PDA, decides current situation by the relative location of the robot, ground borders, ball, and goal. Subsequently, appropriate path and motion set are generated by path planner and motion planner, respectively. It also selects proper vision modules to use according to the detected situation.

In case of the FIRA HuroSot, global vision systems are not approved. Therefore, there are some limitations in the selection of appropriate sensors for the league. Moreover, computing power is also limited by the size of robot body. Therefore, a PDA having CMOS type camera is chosen to decrease the centralized computing cost as a vision processing module. The image processing module, which is implemented within the PDA, is composed with two sub-modules to localization and detection modules like figure 2 Between these two vision processing modules, one is selected according to the state of robot by situation detector, which is also realized in the PDA. Basically, all the used images are captured when the robot is in stable status after completion of procedural behaviors

4.2. Self-localization using landmarks

Figure 3 shows the whole process of self localization. The purpose of this process is to estimate the position of the robot by finding specific landmarks such as corners of the

ground or a goal. If any corner is found in a captured image, relative distance between the robot and the two borders that compose the corner could be computed through equation (9). After calculating relative distances against the two perpendicular borders, the goal direction can be determined using suitable heuristics. For instance, if there was no goal shown on the screen, the robot turns according to the gradient of detected border lines of playground. After finding landmarks such as a corner or a goal, location estimation is performed by the geometrical computation derived from the heuristics of the localization condition. Figure 3 shows how to find appropriate landmarks according to the situation. Localization

Ground Edge exist?

Raise Head

N or

Walk forward Y N

N

+ line, - line both exist?

Line Angle?

Can see Goal?

Y

Y

Calculate Current relative positions and angles to two lines

- /+

Calculate Goal Line Angle

Using these geometrical features, localization could be performed with simple trigonometric equations. Figure 4 shows the preprocessing procedure of selflocalization. If the color detection is only used for finding ground area, the results are strongly dependent on the direction of robot and the light condition of the ground. Due to the size of the playground, the difference of RGB values between near and far position of the ground are large. After the detection of ground area, line detection has to be performed for the self-localization. Sobel mask is used for edge detection in parallel to the RGB based detection, while performing line detection and improving ground detection simultaneously. Using the edge detected image, strong edges are selected according to the predefined threshold. Along those edges, color comparison between upper and lower side of edge is applied. This method decreases the candidates of ground edges and erases the unnecessary edges for the line detection. After removing all unnecessary edges, thinning is applied to the result to diminish search space of line detection, achieved by Hough transformation [12].

1

Turn Left

none

none

Turn Right

left

right Past Turn Direction ?

Turn Left

Turn Right

N

Can see Goal?

Y N 1

Line gradient P to N or N to P? Y Current Direction : Facing Side lines Current Position

Current Direction : Facing Goal Current Position

Calculate Current Position

LocalizationEnd

Figure 3: Self-localization process diagram Using the relationships of landmarks, localization is performed by geometrical equations based on the heuristics of FIRA HuroSot. For the efficiency of calculation, we combined edge detection with color based recognition. With the result, the feature points are derived, and used to calculate relative coordinates of each point against the robot.

4.3. Image preprocessing for border detection In case of FIRA HuroSot, all the image processing is performed upon the image of the playground. Therefore, geometric traits of the ground may be used for localization. The shape of ground is rectangular, and all the objects which can be used as landmarks are laid on that ground.

Figure 4: Preprocessing for localization Through the Hough transform of edges, the characteristic values of border lines such as θ and ρ are derived. Here, θ represents the line angle where ρ represents the distance between the origin and the line in polar coordinates. After finishing Hough transform for all candidates of border lines, the most possible lines are chosen by the heuristic of line length. Using these final lines, intersection point X is calculated as follows:

ρ1 = x cos(θ1 ) + y sin(θ1 ) ρ 2 = x cos(θ 2 ) + y sin(θ 2 )

(1) (2)

ρ1 cos(θ1 ) sin(θ1 ) x = ρ 2 cos(θ 2 ) sin(θ 2 ) y

(3)

A = BX

∴ X = B −1 A

(4)

The parameters required for calculating relational coordinates against the frontal direction of the robot are the height (h) , tilt angle ( θ ), angle of view of camera (Φ x , Φ y ) and the size of image plane ( S x , S y ) . With above

given information, relational perspective coordinates of any feature points in the image are calculated as follows:

4.4. Feature point extraction Basically, to derive the depth information from the captured image, two images from stereo vision system are needed. However, as we are dealing with FIRA HuroSot, there are following three useful traits could be used for estimating depth information from one image: 1. The playground dimensions are predefined. 2. The border lines of the playground are perpendicular to each other. 3. All the elements on the ground such as the goal, ball and opponents exist just on the ground. From these traits, depth information is derived from a single captured image. Figures 5 and 6 show the corresponding relationships between arbitrary point in captured image and real coordinate. The relational coordinates of feature points in the images are obtained through geometrical equations. Camera

Φy

α

( Px , Py )

θy

h Φx

β

y ( xd , yd )

Φy

, β = 90 − α 2 ⎛ P ⎞ ⎛ P 1⎞ θ x = Φ x ⎜⎜ x − ⎟⎟, θ y = Φ y ⎜⎜1 − y ⎟⎟ ⎝ Sx 2 ⎠ ⎝ Sy ⎠

(5)

yd = h tan(α + θ y ), xd = yd tan(θ x )

(7)

α = θt −

(6)

Where all angles are in degrees, ( S x , S y ) and ( Px , Py ) are measured in pixel units, and ( Px , Py ) is the coordinate of feature point on screen in pixel units. The useful features in the images are ball, goal, and the border line of the playground. The former two features are detected by RGB detection module and their positions are assumed by the centroid of points over the threshold. The last feature is derived from edge detection merged process explained in Section 4.3.

4.5. Computation of relative location

In the deliberative layer provided in PDA, self-localization is performed in two ways: Using the relative position of ball and goal, and using the border lines of the playground. Figure 7 shows the geometrical relationships of penalty kick situation where the robot started the match on an arbitrary position. The robot is assumed to face the ball and goal in the same screen. Drg , Drb , θ rg , θ brt are generated through equations (5) to (7). In figure 6, Tpoint is the target point of robot to move for kicking. Dk is the minimum distance for kicking mechanism. Using the given and calculated values so far, θ rg ,θ grb ,θ brt ,θ gbr ,θ brt are derived

Image plane

x Figure 5: Image plane and real coordinate

through trigonometrical calculation. From the result, the angle which robot has to turn, θ turn , θ Dturn are computed

y

as follows:

xl (xd , yd )

yl Φx

−x

+x

Figure 6: X-Y coordinate of Fig. 4.

(8)

θ Dturn = θ gbr − θ brt

(9)

Turning angle, θ turn , walking distance, d walk , forward and turning angle, θ DTurn back in the opposite direction

( px , py )

θx

θ turn = θ rg + θ grb + θ brt

make the robot to face the goal and ball. If the robot has to execute self-localization without being given coordinates of ball, goal and robot itself, the robot needs landmarks to guess its location. If the border of the playground is the only landmark on the screen, the robot has to turn to find specific landmarks for guessing its position. For the efficiency of finding possible landmarks,

y2 − y1 ⎛ x2 − x1 ⎞ x1 − y1 ⎟⎟ x2 − x1 − y y ψ = 1⎠ ⎝ 2 2 ⎛ y2 − y1 ⎞ ⎟⎟ + 1 ⎜⎜ ⎝ x2 − x1 ⎠

the robot must be forced to turn to the direction of finding a goal with less turning angle along with the border gradients.

x y

(10)

θ = atan2⎜⎜

Goal ( xg, yg )

5. EXPERIMENTS

θrgb

Dgb

Ball

Humanoid robots are required to perform a wide variety of tasks in the real world. The proposed scheme was tested through the penalty kick experiment. The robot starts operating from an arbitrary position on the playground. If there is no goal in the screen, it starts to turn to the direction having the best possibility, to find the corner or goal as like in the self-localization process. If a goal is detected before a corner, penalty kick process starts immediately. Figure 9 shows the finite state machine of penalty kick experiment.

( xb , yb )

Dk

θbrt

Drg D rb

θrg

θDturn

Tpo int

d walk

θturn θ grb

sightof robot ( 0, 0)

Robot

Figure 7: Geometrical relationship in penalty kick situation

State:0 Initialization InitDone

State:5 EndTask

NoBall | NoGoal

The intersecting points between borders and image plane like figure 8 (a) are used for computing distance and relative angle between robot and border. In figure 8, P1 and P2 are the intersecting points between the border and the side edges. Using these points on the screen, the distance between that line and the robot could be obtained using equation (10). In equation (10), the coordinates of ( P1 , P2 ) are computed by equations (5) to (7), and mean relative global coordinates against the front side of the robot. θ is an angle between perpendicular line against the robot and end line where ψ is the distance between the robot and end line. P2

P2

P1

P1

Image plane

y Image plane

(a)

x

Robot

(b)

Figure 8: Image plain and real coordinate

State:1 Finding Ball&Goal

Turn Left

Ball Exists Goal Exists

Turn Right

BallX> LeftXofRFoot BallX
NoBall | NoGoal

State:2 Moving to inline path

Stop

|Ball_X-Goal_X|
Walk

State:3 Approaching to the ball

RSideWalk

RBallDist < KickReadyDist

LSideWalk

State:4 Position Tunning

Kick

Timer > WalkTime

Figure 9: Finite state machine for penalty kick The penalty kick motion is performed according to figures 7 and 9. Based on θturn , d walk , θ DTurn calculated by equations (6) to (8) , equations (9) and (10), the robot moves along the perpendicular line against the goal and approach to the ball. After adjusting the position of robot such that the ball is just in front of the right foot of the robot, the robot approaches the ball and kicks it. For this process, robot has to know its initial relative position against the ball and goal through the methods mentioned so far. Figure 10 shows the performed experiments with time step of two seconds. The colors of the ball and goal are assigned to red and blue in this experiment, and the goal patch of 10cm x 10cm is attached on the position of the goal. In figure 10, robot turns to find corner of the ground, and after the self-localization process, moves to the corresponding perpendicular position to kick against goal

and ball, and kick the ball. [3] J.-H. Kim, D.-H. Kim, Y.-J. Kim, K.-H. Park, J.-H. Park, C.-K. Moon, K. T. Seow, and K.-C. Koh, “Humanoid robot Hansaram: Recent progress and development,” J. of Advanced Computational Intelligence & Intelligent Informatics, vol. 8, no. 1, pp. 45-55, Jan. 2004. [4] S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Yokoi, and H. Hirukawa, “Biped walking pattern generation by a simple three-dimensional inverted pendulum model,” Advanced Robotics, vol. 17, no. 2, pp. 131-147, 2003.

Figure 10: Penalty kick snap shots (2 second timestep)

6. CONCLUSION

This paper presented the developments of HanSaRam-VI with regard to its hybrid architecture for vision and behavior scheduler modules in PDA. While the walking trajectory generator is provided in an embedded computer, vision processing, including feature point extraction, relative coordinate computation and self-localization, is implemented in PDA. Therefore, time domain efficiency is obtained. The penalty kick experiment was carried out to demonstrate the efficiency and applicability of the proposed hybrid architecture and vision preprocessing scheme. The observed results were effective and efficient. However, additional research is needed for decreasing the processing time of deliberative layer to enhance the effectiveness of this research. ACKNOWLEDGMENT

[5] J.-H. Park, “Impedance Control for Biped Robot Locomotion,” IEEE Trans. On Robotics and Automotion, vol. 17, no.6, Dec. 2001. [6] B.-J. Lee, Y.-D. Kim, and J.-H. Kim, “Balance control of humanoid robot for HuroSot,” in Proc. of IFAC World Congress, Prague, Czech, July 2005. [7] Y.-D. Kim, B.-J. Lee, J.-K. Yoo, J.-H. Kim, and J.-H. Ryu,“Compensation for the landing impact force of a humanoid robot by time domain passivity approach,” in Proc. of IEEE Int. Conf. on Robotics and Automation, Orlando, FL, May 2006. [8] K. Harada, S. Kajita, F. Kanehiro, K. Fujiwara, K. Kanako, K. Yokoi, and H. Hirukawa, “Real-time planning of humanoid robot`s gait for force controlled manipulation,” in Proc. of IEEE Int. Conf. on Robotics and Automation, vol. 1, New Orleans, LA, Apr. 2004, pp.616-622. [9] Q. Huang, K. Yokoi, S. Kajita, K. Kaneko, H. Arai, N. Koyachi, and K. Tanie, “ Planning walking patterns for a biped robot,” IEEE Trans. on Robotics and Automation, vol. 17, no. 3, pp.280-289, June 2001.

This work was supported by the Ministry of information & Communications, Korea, under the Information Technology Research Center (ITRC) Support Program.

[10] C.F. Marques, and P.U. Lima, “Vision-based selflocalization for soccer robots,” in Proc. of IEEE Int. Conf. on Intelligent Robots and Systems, Takamatsu, Japan, Nov. 2000.

7. REFERENCES

[11] K. Sabe, M.Fukuchi, J.S. Gutmann, T Ohashi, K. Kawamoto, and T. Yoshigahara, “Obstacle avoidance and path planning for humanoid robots using stereo vision,” in Proc. of IEEE Int. Conf. on Robotics and Automation, New Orleans, LA, Apr. 2004.

[1] J. Chestnutt, M. Lau, G. Cheung, J. Kuffner, J. Hodgins, and T. Kanade, “Footstep planning for the honda asimo humanoid,” in Proc. of IEEE Int. Conf. on Robotics and Automations, Barcelona, Spain, Apr. 2005. [2] H.-O. Lim, S.A. Setiawan, A. Takanishi, “Balance and impedance control for biped humanoid robot locomotion,” in Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Maui, Hawaii, USA, Oct. 2001.

[12] L. Iocchi, D. Mastrantuono, and D. Nardi, “A probabilistic approach to hough localization,” in Proc. of IEEE int. Conf. on Robotics and Automation, Seoul, Korea, May 2001.