16th IEEE International Conference on Control Applications Part of IEEE Multi-conference on Systems and Control Singapore, 1-3 October 2007

WeC04.1

Daisy Chaining Based Visual Servo Control 1 Part I: Adaptive Quaternion-Based Tracking Control G. Hu, S. Mehta, N. Gans, and W. E. Dixon Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL 32611-6250 Email: {gqhu, siddhart, ngans, wdixon}@ufl.edu

Abstract— A quaternion-based visual servo tracking controller for a moving six degrees of freedom object is developed based on image feedback from a moving camera. The control objective is for the object to track a desired trajectory determined by a sequence of prerecorded images from a stationary camera. To achieve this result, several technical issues were resolved including: discriminating the relative velocity between the moving camera and the moving object, compensating for the unknown time-varying distance measurement from the camera to the object, relating the unknown attitude of the control object to some measurable signals, and using unit quaternion to formulate the rotation motion and rotation error system. By utilizing multi-view image geometry to develop homography relationships between the moving camera frame and the moving planar patch coordinate systems, the relative velocity issue is resolved. By using the depth ratios obtained from the homography decomposition, the unknown depth information is related to an unknown constant that can be compensated for by a Lyapunov-based adaptive update law. Lyapunov-based methods are provided to prove the adaptive asymptotic tracking result.

I. I NTRODUCTION Recent advances in image processing and computational technology are enabling visual servo control systems to become more prevalent in autonomous vehicle applications. Instead of relying solely on a global positioning system (GPS) or an inertial measurement units (IMU) for navigation, image-based methods are a promising approach to provide autonomous vehicles with position and orientation (pose) information. Specifically, rather than obtain an inertial measurement of an autonomous vehicle, vision systems can be used to recast the navigation and control problem in terms of the image space. In this paper, a trajectory tracking problem is considered for a six degrees of freedom (DOF) planar patch of feature points. Unlike typical visual servo controllers that require either the camera or the target to remain stationary, a unique aspect of the development in this paper is that a moving monocular camera (e.g., a camera mounted on an unmanned air vehicle) is used to provide feedback to a moving autonomous vehicle. The control objective is for the autonomous vehicle (identified by a planar patch of feature 1 This research is supported in part by the NSF CAREER award CMS0547448, AFOSR contract numbers F49620-03-1-0381 and F49620-03-10170, AFRL contract number FA4819-05-D-0011, research grant No. US3715-05 from BARD, the United States - Israel Binational Agricultural Research and Development Fund at the University of Florida, and funding from DARPA (Approved for Public Release, Distribution Unlimited).

1-4244-0443-6/07/$20.00 ©2007 IEEE.

points) to track the pose of a desired vehicle trajectory that is encoded by a prerecorded video obtained from a fixed camera (e.g., a camera mounted on a satellite, a camera mounted on a building). Several challenges must be resolved to achieve this unexplored control objective. The relative velocity between the moving feature point patch and the moving camera presents a significant challenge. By using a daisy-chaining (e.g., [11]) approach, Euclidean homography relationships between different camera frames and the feature point patch are developed. These homographies are used to relate coordinate frames attached to the moving camera, the current patch, and the desired patch trajectory specified by an a priori sequence of images. Another challenge is that for general six DOF motion by both the camera and the planar patch, the normal to planar patch is unknown. By decomposing the homography relationships, the normal to the moving feature point patch can be obtained. Likewise, the distance between the moving camera, the moving planar patch, and a reference patch are unknown. By using the depth ratios obtained from the homography decomposition, the unknown distance is related to an unknown constant parameter. A Lyapunovbased adaptive control law is designed to compensate for the unknown constant parameter. The moving camera could be attached to a remotely piloted vehicle with arbitrary rotations, this requires a parameterization that is valid over a large (possibly unbounded) domain. Additionally, since this work is motivated by problems in the aerospace community, homography-based visual servo control techniques (e.g., [2], [4], [9]) are combined with quaternion-based control methods (e.g., [3], [7]) to facilitate rotations beyond one revolution. By using the quaternion parameterization, the resulting closed-loop rotation error system can be stabilized by a proportional rotation controller combined with a feedforward term that is a function of the desired trajectory. II. P ROBLEM S CENARIO Over the past decade, a variety of visual servo controllers have been addressed for both camera-to-hand and camerain-hand configurations (e.g., see [1], [6], [8], [14], etc.). For visual servo control applications that exploit either of these camera configurations, either the target or the camera is required to remain stationary. In contrast to typical camerato-hand or camera-in-hand visual servo control configurations, a moving airborne monocular camera (e.g., a camera

1474

WeC04.1

attached to a remote controlled aircraft, a camera mounted on a satellite) is used in [11] to provide pose measurements of a moving sensorless unmanned ground vehicle (UGV) relative to a goal configuration. The result in [11] is restricted to three degrees of freedom, and the rotational error system is encoded by Euler angle-axis parameterization..

Motion R (t), xf (t) R∗ (t), x∗f (t) Rr (t), xf r (t) 0 0 R (t), xf r (t) ∗ ∗ Rr , xf r Rrd (t), xf rd (t)

TABLE I C OORDINATE FRAMES RELATIONSHIPS

Reference R Camera

I

Current Camera

I

(Rrd , xfrd)

(R*r,x*fr) (R*,x*f )

s2i (R,xf)

F*

d

snapshots of the planar patches associated with F and F ∗ , respectively. The a priori motion of Fd represents the desired trajectory of the coordinate system F, where F and Fd are attached to the same object but at different points in time. The camera attached to IR is a different camera (with different calibration parameters) as the camera attached to I. The problem considered in this paper is to develop a kinematic controller for the object attached to F so that the timevarying rotation and translation of F converges to the desired time-varying rotation and translation of Fd , where the motion of F is determined from the time-varying overhead camera attached to I.

d*r

d*

Reference Object

π*

s1i

ent Curr h Patc

s1i π

Desired Patch Trajectory

F

Fig. 1.

Frames F to I in I F ∗ to I in I I to IR F to IR in IR F ∗ to IR in IR Fd to IR in IR

Fd πd

III. G EOMETRIC M ODEL

Geometric Model.

Consider a stationary coordinate frame IR that is attached to a camera and a time-varying coordinate frame Fd that is attached to some object (e.g., an aircraft, a ground vehicle, a marine vessel) as depicted in Fig. 1. The object is identified in an image by a collection of feature points that are assumed (without loss of generality2 ) to be coplanar and non-colinear (i.e., a planar patch of feature points). The camera attached to IR a priori records a series of snapshots (i.e., a video) of the motion of the coordinate frame Fd until Fd comes to rest (or the video stops recording). A stationary coordinate frame F ∗ is attached to another planar patch of feature points that are assumed to be visible in every frame of the video recorded by the camera. For example, the camera attached to IR is on-board a “stationary” satellite that takes a series of snapshots of the relative motion of Fd with respect to F ∗ . Therefore, the desired motion of Fd can be encoded as a series of relative translations and rotations with respect to the stationary frame F ∗ a priori. Spline functions or filter algorithms can be used to generate a smooth desired feature point trajectory [2]. Consider a time-varying coordinate frame I that is attached to a camera (e.g., a camera attached to a remote controlled aircraft) and a time-varying coordinate frame F that is attached to the current pose of the planar patch as depicted in Fig. 1. The camera attached to I captures

The relationships between the coordinate systems are 0 as follows (also see Table I): R (t), R∗ (t), Rr (t), R (t), ∗ Rrd (t), Rr ∈ SO(3) denote the rotation from F to I, F ∗ to I, I to IR , F to IR , Fd to IR , and F ∗ to IR , respectively, xf (t), x∗f (t) ∈ R3 denote the respective timevarying translation from F to I and from F ∗ to I with coordinates expressed in I, and xf r (t), x0f r (t), xf rd (t), x∗f r ∈ R3 denote the respective constant translation from I to IR , F to IR , Fd to IR , and from F ∗ to IR with coordinates expressed in IR . From Fig. 1, the translation 0 x0f r (t) and the rotation R (t) can be expressed as x0f r R

= x∗f r + Rr∗ R∗T (xf − x∗f ) = Rr∗ R∗T R.

(1)

As illustrated in Fig. 1, π, π d and π∗ denote the planes of feature points associated with F, Fd , and F ∗ , respectively. s1i ∈ R3 ∀i = 1, 2, · · · , n (n ≥ 4) denotes the constant Euclidean coordinates of the i-th target point in F (and also Fd ), and s2i ∈ R3 ∀i = 1, 2, · · · , n denotes the constant Euclidean coordinates of the i-th feature point in F ∗ . From the geometry between the coordinate frames depicted in Fig. 1, the following relationships can be developed m ¯ i = xf + Rs1i m ¯ ∗ri

2 Image

processing techniques can often be used to select coplanar and non-colinear feature points within an image. However, if four coplanar target points are not available then the subsequent development can also exploit the virtual parallax method [10] where the non-coplanar points are projected onto a virtual plane.

0

=

x∗f r

+ Rr∗ s2i m ¯ ∗i = x∗f

m ¯ rdi = xf rd + Rrd s1i 0

m ¯i =

x0f r

+ R∗ s2i .

0

+ R s1i

(2) (3) (4)

¯ ∗i (t) ∈ R3 denote the Euclidean coIn (2)-(4), m ¯ i (t), m ordinates of the feature points on π and π ∗ , respectively,

1475

WeC04.1

expressed in I as m ¯ i (t) , m ¯ ∗i (t) 0

,

£ £

xi (t) yi (t) zi (t) x∗i (t)

yi∗ (t)

zi∗ (t)

¤T

¤T

(5) ,

(6)

m ¯ i (t), m ¯ rdi (t) ∈ R3 denote the actual and desired timevarying Euclidean coordinates, respectively, of the feature points on π expressed in IR as £ 0 ¤T 0 0 0 m ¯ i (t) , (7) xi (t) yi (t) zi (t) £ ¤T xrdi (t) yrdi (t) zrdi (t) , (8) m ¯ rdi (t) ,

and m ¯ ∗ri ∈ R3 denotes the constant Euclidean coordinates of the feature points on the plane π ∗ expressed in IR as £ ¤T ∗ ∗ zri m ¯ ∗ri , x∗ri yri . (9) After some algebraic manipulation, the expressions in (2)-(4) can be rewritten as m ¯ ∗i = x ¯n + Rn m ¯i ¯m ¯f + R ¯ ∗i m ¯i = x

(10)

¯ rd m m ¯ rdi = x ¯f rd + R ¯ ∗ri

m ¯ ∗ri = xf r + Rr m ¯ ∗i

0

m ¯ i = xf r + Rr m ¯ i,

(11) (12)

¯ (t), R ¯ rd (t), Rr (t) ∈ SO (3) and x where Rn (t), R ¯n (t), x ¯f (t), x ¯f rd (t), xf r (t) ∈ R3 are new rotational and translational variables, respectively, defined as3 Rn ¯ Rrd

x ¯n x ¯f x ¯f rd xf r

= R∗ RT = Rrd Rr∗T

¯ = RR∗T R Rr = Rr∗ R∗T

= x∗f − Rn (xf − R (s2i − s1i )) ¡ ¢ ¯ x∗f + R∗ (s2i − s1i ) = xf − R ¢ ¡ ¯ rd x∗f r + Rr∗ (s2i − s1i ) = xf rd − R = x∗f r − Rr x∗f = x0f r − Rr xf .

d∗ (t) = n∗T m ¯ ∗i

Remark 1: As in [2], the subsequent development requires that the constant rotation matrix Rr∗ be known. The constant rotation matrix Rr∗ can be obtained a priori using various methods (e.g., a second camera, Euclidean measurements). The subsequent development also assumes that the difference between the Euclidean distances (s2i − s1i ) is a constant ∀i = 1, ..., n. This assumption means that the unknown constant Euclidean distances s2i and s1i are different by some unknown scalar multiple. While there are many practical applications that satisfy this assumption (e.g., a simple scenario is that the objects attached to F and F ∗ are identical), the assumption is generally restrictive and is the focus of future research. IV. E UCLIDEAN R ECONSTRUCTION

(13)

(14) (15) (16) (17)

To facilitate the development of a relationship between the actual Euclidean translation of F to the Euclidean translation that is reconstructed from the image information, the following projective relationships are developed: d(t) = nT m ¯i

Based on (18), the relationships in (10)-(12) can be expressed as ³ x ¯n T ´ m ¯ ∗i = Rn + (19) ¯i n m d ´ ³ ¯f ∗T ¯+x R n (20) m ¯ ∗i m ¯i = ∗ d µ ¶ ¯f rd ∗T ¯ rd + x R n (21) m ¯ ∗ri m ¯ rdi = d∗r r µ ¶ xf r n∗T m ¯ ∗ri = Rr + (22) m ¯ ∗i d∗ µ ¶ 0 xf r nT m ¯i = Rr + (23) m ¯ i. d

d∗r = n∗T ¯ ∗ri , (18) r m

where d(t) ∈ R represents the distance from the origin of I to π along the unit normal (expressed in I) to π denoted as n(t) ∈ R3 , d∗ (t) ∈ R represents the distance from the origin of I to π ∗ along the unit normal (expressed in I) to π ∗ denoted as n∗ (t) ∈ R3 , and d∗r ∈ R represents the distance from the origin of IR to π ∗ along the unit normal (expressed in IR ) to π ∗ denoted as n∗r ∈ R3 where n∗ (t) = RrT (t)n∗r . In (18), d(t), d∗ (t), d∗r > ε for some positive constant ε ∈ R. 3 Note that R (t), R(t) ¯ ¯ rd (t) in (13) are the rotation matrices and R n between F and F ∗ , F ∗ and F, and F ∗ and Fd , respectively, but x ¯n (t), x ¯f (t) and x ¯f rd (t) in (14)-(16) are not the translation vectors between the corresponding coordinate frames. However, this will not affect the following controller design because only the rotation matrices will be used in the controller development.

The relationships given by (19)-(23) provide a means to quantify a translation and rotation error between the different coordinate systems. Since the position and orientation (i.e., pose) of F, Fd , and F ∗ cannot be directly measured, a Euclidean reconstruction is developed to obtain the pose error by comparing multiple images acquired from the hovering monocular vision system. To facilitate the subsequent development, the normalized Euclidean coordinates of the feature points in π and π∗ can be expressed in terms of I as mi (t) ∈ R3 and m∗i (t) ∈ R3 , respectively, as mi ,

m ¯i zi

m∗i ,

m ¯ ∗i . zi∗

(24)

Similarly, the normalized Euclidean coordinates of the feature points for the current, desired, and reference image can 0 be expressed in terms of IR as mi (t), mrdi (t), m∗ri ∈ R3 , respectively, as 0

mi (t) ,

0

m ¯ i (t) 0 zi (t)

mrdi (t) ,

m ¯ rdi (t) zrdi (t)

m∗ri ,

m ¯ ∗ri ∗ . (25) zri

From the expressions given in (20) and (24), the rotation and translation between the coordinate systems F and F ∗ , between F ∗ and Fd , and between I and IR can now be related in terms of the normalized Euclidean coordinates as follows: ¡ ¢ ¯ + xh n∗T m∗i , (26) mi = αi R

1476

m∗i =

¢ 1 ¡ Rn + xnh nT mi αi

(27)

WeC04.1

¡ ¢ ∗ ¯ rd + xhrd n∗T mrdi = αrdi R mri , r ¡ ¢ m∗ri = αri Rr + xhr n∗T m∗i ,

(28) (29)

where αi (t), αrdi (t), αri (t) ∈ R denote depth ratios defined as z∗ z∗ z∗ αi = i αrdi = ri αri = ∗i , zi zrdi zri and xh (t), xnh (t), xhrd (t), xhr (t) ∈ R3 denote scaled translation vectors that are defined as x ¯f x ¯n xh = xnh = (30) ∗ d d x ¯f rd xf r xhr = ∗ . xhrd = d∗r d Since the normalized Euclidean coordinates in (26)-(29) can not be directly measured, the following relationships (i.e., the pin-hole camera model) are used to determine the normalized Euclidean coordinates from pixel information pi prdi

= A1 mi = A2 mrdi

p∗i = A1 m∗i p∗ri = A2 m∗ri ,

respectively, where ui (t), vi (t) , u∗i (t), vi∗ (t) ∈ R. Similarly, prdi (t) and p∗ri ∈ R3 represent the image-space coordinates of the Euclidean features on πd and π ∗ expressed in terms of IR as £ £ ¤T ¤T ∗ 1 p∗ri , u∗ri vri prdi , urdi vrdi 1 (34) ∗ ∈ R. By using respectively, where urdi (t), vrdi (t), u∗ri , vri (26)-(29) and (31)-(34), the following relationships can be developed: ¡ ¡ ¢ ¢ ¯ + xh n∗T A−1 p∗ pi = αi A1 R i 1 {z } | (35) G ¢ ¢ 1 ¡ ¡ pi A1 Rn + xnh nT A−1 1 αi | {z } Gn ¡ ¡ ¢ −1 ¢ ∗ ¯ rd + xhrd n∗T A2 pri prdi = αrdi A2 R r {z } | Grd ¡ ¡ ¢ ¢ ∗ pi , p∗ri = αri A2 Rr + xhr n∗T A−1 1 {z } | Gr

zi (t) can be determined as 0 zi (t) µ ¶ 0 £ ¤ zi n∗T m∗i T = 0 0 1 Rr + xhr αi T n mi . (40) zi n mi

where the inverse of the ratio

(31) (32)

where A1 , A2 ∈ R3×3 are known, constant, and invertible intrinsic camera calibration matrices of the current camera and the reference camera, respectively. In (31) and (32), pi (t) and p∗i (t) ∈ R3 represent the image-space coordinates of the Euclidean feature points on π and π∗ expressed in terms of I as £ £ ¤T ¤T pi , ui vi 1 p∗i , u∗i vi∗ 1 , (33)

p∗i =

to a scalar multiple. Various techniques can be used (e.g., see [5], [15]) to decompose the Euclidean homographies, to obtain αi (t) , αrdi (t), αri (t), xh (t), xnh (t), xhrd (t), ¯ (t), Rn (t), R ¯ rd (t), Rr (t), n∗ (t), n∗r , n (t). Given xhr (t), R that the constant rotation matrix Rr∗ is assumed to be known, ¯ rd (t) and Rr (t) in (13) can be used to the expressions for R determine Rrd (t) and R∗ (t). Once R∗ (t) is determined, the ¯ expression for R(t) in (13) can be used to determine R(t). ∗ Also, once Rr , R∗T (t), and R (t) have been determined, (1) can be used to determine R0 (t). Since Rr (t), xhr (t), αi (t), n∗ (t), n∗r , n (t), m∗i (t), and mi (t) can be determined, the 0 following relationship can be used to determine mi (t): µ ¶ 0 zi n∗T m∗i T n (39) mi , mi = 0 Rr + xhr αi T zi n mi

(36)

(37)

(38)

where G (t), Gn (t), Grd (t), Gr (t) ∈ R3×3 denote projective homographies. Sets of linear equations can be developed from (35)-(38) to determine the projective homographies up

V. C ONTROL O BJECTIVE

The control objective is for a planar patch (e.g., a UGV or a UAV) to track a desired trajectory that is determined by a sequence of images taken by a fixed reference camera. This objective is based on the assumption that the linear and angular velocities of the camera are control inputs that can be independently controlled (i.e., unconstrained motion) and that the reference and desired cameras are calibrated (i.e., A1 and A2 are known). The control objective can 0 be stated as m ¯ i (t) → m ¯ rdi (t) (i.e., the Euclidean feature points on π track the corresponding feature points on π d ). Equivalently, the control objective can also be stated in terms 0 of the rotation and translation of the object as xf r (t) → xf rd (t) and R0 (t) → Rrd (t). As stated previously, R0 (t) and Rrd (t) can be computed by decomposing the projective homographies in (35)-(38) and using (1). Once these rotation matrices have been determined, the unit quaternion parameterization is used to describe the rotation matrix. This parameterization facilitates the subsequent problem formulation, control development, and stability analysis since the unit quaternion provides a globally nonsingular parameterization of the corresponding rotation matrices. The unit quaternion is a four dimensional vector which can be defined as [3] £ £ ¤T ¤T q , q0 qvT qv , qv1 qv2 qv3 , (41)

where q0 (t), qvi (t) ∈ R ∀i = 1, 2, 3 satisfy the following nonlinear constraint q T q = 1. (42)

Given the rotation matrices R0 (t) and Rrd (t), the corresponding unit quaternions q (t) and qd (t) can be calculated by using the numerically robust method presented in [7] and

1477

WeC04.1

[13] based on the corresponding relationships ¡ ¢ (43) R0 = q02 − qvT qv I3 + 2qv qvT + 2q0 qv× ¡ 2 ¢ × T T Rrd = q0d − qvd qvd I3 + 2qvd qvd + 2q0d qvd (44)

where I3 is the 3 × 3 identity matrix, and the notation qv× (t) denotes the following skew-symmetric form of the vector qv (t): ⎡ ⎡ ⎤ ⎤ 0 −qv3 qv2 qv1 0 −qv1 ⎦ , ∀qv = ⎣ qv2 ⎦ . (45) qv× = ⎣ qv3 −qv2 qv1 0 qv3 To quantify the rotation error between the feature points on π and π d , the error between rotation matrices R0 (t) and Rrd (t) is defined as ¡ ¢ ˜ = R0T Rrd = q˜02 − q˜vT q˜v I3 + 2˜ R qv q˜vT − 2˜ q0 q˜v× , (46)

where the error quaternion q˜(t) = (˜ q0 (t), q˜vT (t))T is defined as ¸ ∙ ¸ ∙ q0 q0d + qvT qvd q˜0 = . (47) q˜ = q˜v q0d qv − q0 qvd + qv× qvd

Since q˜(t) is a unit quaternion, (46) can be used to quantify the rotation tracking objective as ˜ → I3 k˜ qv (t)k → 0 =⇒ R(t)

as t → ∞.

(48)

The translation error, denoted by e(t) ∈ R3 , is defined as [2], [9] e = me − med (49) where me (t), med (t) ∈ R are defined as 3

me med



0

0

¸T

xi yi 0 ln(zi ) 0 0 zi zi h xrdi yrdi iT ln(zrdi ) = . zrdi zrdi =

(50)

ke(t)k → 0 as t → ∞.

where vc (t) ∈ R3 denotes the linear velocity of π expressed in F, ω c (t) ∈ R3 denotes the angular velocity of π expressed 0 in F, and the auxiliary measurable terms Lv (t), Lvd (t), 0 Lvω (t) ∈ R3×3 are defined as ⎡ 0 ⎤ ⎡ xi xrdi ⎤ 1 0 − 1 0 − 0 ⎥ ⎢ zi ⎥ z ⎥ ⎢ ⎢ 0 0 yrdi rdi ⎥ ⎥ Lvd , ⎢ Lv , ⎢ y 0 1 − ⎢ 0 1 − i ⎥ ⎦ ⎣ 0 zrdi ⎣ zi ⎦ 0 0 1 0 0 1 ⎤ ⎡ 0 0 0 0 xi yi yi x2 −1 − 0i2 02 0 ⎥ ⎢ zi zi ⎥ ⎢ zi 0 0 0 ⎥ ⎢ 0 y 02 xy x ⎥ ⎢ Lvω , ⎢ 1 + i0 2 − i0 2i − 0i ⎥ . ⎢ z z zi ⎥ i i ⎥ ⎢ 0 0 ⎦ ⎣ xi yi − 0 0 0 zi zi

B. Closed-Loop Error System

Based on the open-loop rotation error system in (52) and the subsequent Lyapunov-based stability analysis, the angular velocity controller is designed as ˜ cd , ω c = −Kω q˜v + Rω

·

(51)

·

q˜v

1 T q˜ Kω q˜v 2 v ¢ 1¡ = − q˜0 I3 + q˜v× Kω q˜v . 2 =

0

vc

A. Open-Loop Error System

=

αri zi 0 T 0 −1 R Lv [Kv e αi ziµ 0

Based on (46) and (47), the open-loop rotation error system can be developed as [3] ¸³ ∙ ´ · 1 −˜ qvT ˜ q˜ = − Rω ω , (52) c cd 2 q˜0 I3 + q˜v×

where ω cd (t) denotes the angular velocity of π d expressed in Fd that can be calculated as [3]

(53)

(56)

Based on (54), the translation control input vc (t) is designed as

VI. C ONTROL D EVELOPMENT

× q˙vd , ω cd = 2(q0d q˙vd − qvd q˙0d ) − 2qvd

(55)

where Kω ∈ R3×3 denotes a diagonal matrix of positive constant control gains. From (52) and (55), the rotation closed-loop error system can be determined as q˜0

Based on (48) and (49), the subsequent control development targets the following objectives: k˜ qv (t)k → 0 and

¡ ¢T ¡ ¢T T T where q0d (t), qvd (t) , q˙0d (t), q˙vd (t) are assumed to be bounded; hence, ω cd (t) is also bounded. The open-loop translation error system can be derived as 0 zi αi 0 0 ∗ ∗ 0 zri e˙ = − 0 L R vc + zri Lvω R ω c (54) zi αri v ¶ µ d prdi ∗ −zri αrdi Lvd A−1 2 dt αrdi

0

∗ Lvω R ω c − αrdi Lvd A−1 +ˆ zri 2

d dt

µ

(57) ¶¶¸ prdi , αrdi

where Kv ∈ R3×3 denotes a diagonal matrix of positive con∗ stant control gains. In (57), the parameter estimate zˆri (t) ∈ R ∗ for the unknown constant zri is defined as µ ¶¶ µ · 0 0 prdi −1 d ∗ T zˆri = γe Lvω R ω c − αrdi Lvd A2 , (58) dt αrdi where γ ∈ R denotes a positive constant adaptation gain. By using (54) and (57), the translation closed-loop error system

1478

WeC04.1

is

µ ¶¶ µ 0 0 d prdi ∗ ∗ zri e˙ = −Kv e+ Lvω R ω c − αrdi Lvd A−1 , z˜ri 2 dt αrdi (59) ∗ (t) ∈ R denotes the following parameter estimation where z˜ri error: ∗ ∗ ∗ = zri − zˆri . (60) z˜ri

homography relationships between different camera frames and the planar patch. Lyapunov-based methods are used to prove the asymptotic tracking result. Future efforts will consider uncertainties in the camera calibration, weakening the assumptions imposed on the feature points, and using a moving camera to record the sequence of desired images instead of a fixed camera. R EFERENCES

C. Stability Analysis Theorem 1: The controller given in (55) and (57), along with the adaptive update law in (58) ensures asymptotic tracking in the sense that k˜ qv (t)k → 0, ke(t)k → 0, as t → ∞. (61) Proof: Let V (t) ∈ R denote the following differentiable non-negative function (i.e., a Lyapunov candidate): V = q˜vT q˜v + (1 − q˜0 )2 +

∗ zri 1 ∗2 eT e + z˜ . 2 2γ ri

(62)

The time-derivative of V (t) can be determined as ¡ ¢ V˙ = −˜ qvT q˜0 I3 + q˜v× Kω q˜v − (1 − q˜0 )˜ qvT Kω q˜v −eT K µv e

¶¶ µ 0 0 d prdi ∗ Lvω R ω c − αrdi Lvd A−1 z˜ri 2 dt αrdi µ ¶¶ µ 0 0 d prdi ∗ T −˜ zri e Lvω R ω c − αrdi Lvd A−1 2 dt αrdi ¡ ¢ T × = −˜ qv q˜0 I3 + q˜v + (1 − q˜0 )I3 Kω q˜v − eT Kv e +eT

= −˜ qvT Kω q˜v − eT Kv e,

(63)

where (56) and (58)-(60) were utilized. Based on (62) and ∗ (63), e(t), q˜v (t), q˜0 (t), z˜ri (t) ∈ L∞ and e(t), q˜v (t) ∈ ∗ L2 . Since z˜ri (t) ∈ L∞ , it is clear from (60) that ∗ zˆri (t) ∈ L∞ . Based on the fact that e(t) ∈ L∞ , 0 (49) and (50) can be used to prove that mi (t) ∈ L∞ , 0 0 0 and then Lv (t), Lv−1 (t) , Lvω (t) ∈ L∞ . Based on the fact that q˜v (t) ∈ L∞ and ω cd (t) is a bounded function, (55) can be used to conclude that ω c (t) ∈ L∞ . 0 0 0 0 ∗ Since zˆµri (t), ¶ e(t), mi (t), Lv (t), Lv−1 (t) , Lvω (t) ∈ L∞ d prdi and is bounded, (57) can be utilized to prove that dt αrdi vc (t) ∈ L∞ . From the previous results, (52)-(54) can be used ·

to prove that e(t), ˙ q˜v (t) ∈ L∞ . Since e(t), q˜v (t) ∈ L∞ ∩L2 , ·

and e(t), ˙ q˜v (t) ∈ L∞ , we can utilize a corollary to Barbalat’s Lemma [12] to conclude the result given in (61). VII. C ONCLUSIONS

[1] P. K. Allen, A. Timcenko, B. Yoshimi, and P. Michelman, “Automated Tracking and Grasping of a Moving Object with a Robotic Hand-Eye System,” IEEE Transaction on Robotics and Automation, Vol. 9, No. 2, pp. 152-165, (1993). [2] J. Chen, D. M. Dawson, W. E. Dixon, and A. Behal, “Adaptive Homography-Based Visual Servo Tracking for Fixed and Camera-inHand Configurations,” IEEE Transactions on Control Systems Technology, Vol. 13, No. 5, pp. 814-825, (2005). [3] W. E. Dixon, A. Behal, D. M. Dawson, and S. Nagarkatti, Nonlinear Control of Engineering Systems: A Lyapunov-Based Approach, Birkhäuser Boston, 2003. [4] Y. Fang, W. E. Dixon, D. M. Dawson, and P. Chawda, “HomographyBased Visual Servoing of Wheeled Mobile Robots,” IEEE Transactions on Systems, Man, and Cybernetics -Part B: Cybernetics, Vol. 35, No. 5, pp. 1041-1050, (2005). [5] O. Faugeras and F. Lustman, “Motion and Structure From Motion in a Piecewise Planar Environment”, International Journal of Pattern Recognition and Artificial Intelligence, Vol. 2, No. 3, pp. 485-508, (1988). [6] G. D. Hager, W.-C. Chang, and A. S. Morse, “Robot Hand-Eye Coordination Based on Stereo Vision,” IEEE Control Systems Magazine, Vol. 15, No. 1, pp. 30-39, (1995). [7] G. Hu, W. E. Dixon, S. Gupta, and N. Fitz-coy, “A Quaternion Formulation for Homography-based Visual Servo Control,” Proceedings of the IEEE International Conference on Robotics and Automation, Orlando, Florida, 2006, pp. 2391-2396. [8] S. Hutchinson, G. D. Hager, and P. I. Corke, “A Tutorial on Visual Servo Control,” IEEE Transactions on Robotics and Automation, Vol. 12, No. 5, pp. 651-670, (1996). [9] E. Malis, F. Chaumette, and S. Boudet, “2D 1/2 Visual Servoing,” IEEE Transaction on Robotics and Automation, Vol. 15, No. 2, pp. 234-246, (1999). [10] E. Malis and F. Chaumette, “2 1/2 D Visual Servoing with Respect to Unknown Objects through a New Estimation Scheme of Camera Displacement,” International Journal of Computer Vision, Vol. 37, No. 1, pp. 79-97, (2000). [11] S. Mehta, W. E. Dixon, D. MacArthur, C. D. Crane, “Visual Servo Control of an Unmanned Ground Vehicle via a Moving Airborne Monocular Camera,” Proceedings of the IEEE American Control Conference, Minneapolis, Minnesota, 2006, pp. 5276-5211. [12] J. J. Slotine and W. Li, Applied Nonlinear Control, Englewood Cliff, NJ: Prentice Hall, Inc., 1991. [13] M. Shuster, “A Survey of Attitude Representations,” The Journal of the Astronautical Sciences, Vol. 41, No. 4, pp. 439-518, (1993). [14] S. Wiiesoma, D. Wolfe, and R. Richards, “Eye-to-hand Coordination for Vision-Guided Robot Control Applications,” International Journal of Robotics Research, Vol. 12, No. 1, pp. 65-78, (1993). [15] Z. Zhang and A. R. Hanson, “Scaled Euclidean 3D Reconstruction Based on Externally Uncalibrated Cameras,” IEEE Symposium on Computer Vision, 1995, pp. 37-42.

A collaborative adaptive quaternion-based visual servo controller is developed to enable a planar patch of feature points to track a desired trajectory determined by a sequence of prerecorded images. In contrast to typical camera-tohand or camera-in-hand visual servo control configurations, the controller in this paper is developed using a moving on-board camera viewing a moving target to obtain feedback signals. A daisy chaining method is used to develop

1479

Daisy Chaining Based Visual Servo Control Part I - IEEE Xplore

Email: {gqhu, siddhart, ngans, wdixon}@ufl.edu. Abstract—A quaternion-based visual servo tracking con- troller for a moving six degrees of freedom object is ...

300KB Sizes 1 Downloads 490 Views

Recommend Documents

Daisy Chaining Based Visual Servo Control Part II - IEEE Xplore
Email: {gqhu, ngans, siddhart, wdixon}@ufl.edu. Abstract— In this paper, the open problems and applications of a daisy chaining visual servo control strategy ...

A Geometric Approach to Visual Servo Control in the ... - IEEE Xplore
University of Florida. Shalimar, FL-32579, USA. J. W. Curtis. Munitions Directorate. Air Force Research Laboratory. Eglin AFB, FL-32542, USA. Abstract—In this paper, we formulate a visual servo control problem when a reference image corresponding t

A Daisy-Chaining Visual Servoing Approach with ...
Following the development in Section 2.2 and 2.3, relationships can be obtained to determine the homographies and depth ratios as4 pi = αi (A ( ¯R + xhn∗T) ...

I iJl! - IEEE Xplore
Email: [email protected]. Abstract: A ... consumptions are 8.3mA and 1.lmA for WCDMA mode .... 8.3mA from a 1.5V supply under WCDMA mode and.

Proto-Object Based Rate Control for JPEG2000: An ... - IEEE Xplore
Abstract—The JPEG2000 system provides scalability with respect to quality, resolution and color component in the transfer of images. However, scalability with ...

Magnetic Induction Micromachine—Part I: Design and ... - IEEE Xplore
formance increase in MEMS electromagnetic machines studied to date. [1435]. Index Terms—Magnetic induction motor, micromotor, power. MEMS, magnetic ...

Magnetic Induction Micromachine—Part I: Design and ... - IEEE Xplore
Design and Analysis. Hur Koser, Member, IEEE, and Jeffrey H. Lang, Fellow, IEEE. Abstract—Most microscale electric and magnetic machines studied in the last ...

Proto-Object Based Rate Control for JPEG2000: An ... - IEEE Xplore
Constraints inherent in a modern visual data transmission system, such as heterogeneous network, varying connection quality, or the need to operate on a variety of devices with a wide range of capabilities, motivate an intense worldwide research effo

Dynamic Interactions between Visual Experiences ... - IEEE Xplore
Abstract—The primary aim of this special session is to inform the conference's interdisciplinary audience about the state-of-the-art in developmental studies of ...

Codebook-Based Opportunistic Interference Alignment - IEEE Xplore
May 9, 2014 - based on the exiting zero-forcing receiver. We first propose a codebook-based OIA, in which the weight vectors are chosen from a predefined ...

lte part ii: radio access - IEEE Xplore
was to reduce user plane latency to less than 10 ms and .... editorial team would like to thank our companies for pro- .... tions and real-time software. He is the ...

Computationally Efficient Template-Based Face ... - IEEE Xplore
head poses, illuminations, ages and facial expressions. Template images could come from still images or video frames. Therefore, measuring the similarity ...

Noniterative Interpolation-Based Super-Resolution ... - IEEE Xplore
Noniterative Interpolation-Based Super-Resolution. Minimizing Aliasing in the Reconstructed Image. Alfonso Sánchez-Beato and Gonzalo Pajares, Member, ...

Content-Based Copy Retrieval Using Distortion-Based ... - IEEE Xplore
very large databases both in terms of quality and speed. ... large period, refers to a major historical event. ... that could be exploited by data mining methods.

IEEE Photonics Technology - IEEE Xplore
Abstract—Due to the high beam divergence of standard laser diodes (LDs), these are not suitable for wavelength-selective feed- back without extra optical ...

Multi-Reference Visual Servo Control of an Unmanned ...
Department of Mechanical and Aerospace Engineering, University of Florida, ... This research is supported in part by the NSF CAREER AWARD 0547448, ...... Navigation, and Control Conference, Keystone, Colorado, AIAA 2006-6718, 2006.

wright layout - IEEE Xplore
tive specifications for voice over asynchronous transfer mode (VoATM) [2], voice over IP. (VoIP), and voice over frame relay (VoFR) [3]. Much has been written ...

Device Ensembles - IEEE Xplore
Dec 2, 2004 - time, the computer and consumer electronics indus- tries are defining ... tered on data synchronization between desktops and personal digital ...

wright layout - IEEE Xplore
ACCEPTED FROM OPEN CALL. INTRODUCTION. Two trends motivate this article: first, the growth of telecommunications industry interest in the implementation ...

Teach by Zooming Visual Servo Control for an ...
AIAA Guidance, Navigation, and Control Conference and Exhibit ..... In (22), ˆHd (t) ∈ R3x3 denotes the following estimated Euclidean homography:11 ..... Improvements in the Stability Analysis of a New Class of Model-Free Visual. Servoing ...

visual servo control of nonholonomic mobile robots
and/or mechanical constraints of the robotic platform. Consider the system of Fig ..... Transactions on Robotics and Automation 8, 313–326. Samson, Claude and ...

Optimizing Binary Fisher Codes for Visual Search - IEEE Xplore
The Institute of Digital Media, Peking University, Beijing, China. {zhew,lingyu,linjie,cjie,tjhuang,wgao}@pku.edu.cn. Fisher vectors (FV), a global representation obtained by aggregating local invari- ant features (e.g., SIFT), generates the state-of

Toward “Pseudo-Haptic Avatars”: Modifying the Visual ... - IEEE Xplore
In our experimental setup, participants could watch their self-avatar in a virtual environment in mirror mode while performing a weight lifting task. Users could ...

Cluster Space Control of Autonomous Surface Vessels ... - IEEE Xplore
a single robot system including redundancy, coverage and flexibility. One of the ... surface vessels consisting of 2 or 3 robots and with varying implementations ... flexible and mobile perimeter formed by the ASV cluster or to detect a threat and ..