Investigation of Morphology and Control in Biped Locomotion Dissertation zur ¨ Erlangung der naturwissenschaftlichen Doktorwurde (Dr. sc. nat.) vorgelegt der Mathematisch-naturwissenschaftlichen Fakult¨at der ¨ Universit¨at Zurich von Chandana Paul aus den U.S.A.
Begutachtet von Prof. Dr. Rolf Pfeifer Prof. Dr. Henrik Lund Dr. Andre Seyfarth
¨ Zurich 2004
Responsible MNF Faculty Members: Prof. Dr. Rolf Pfeifer Prof. Dr. Rodney Douglas
Abstract This thesis focuses on investigating the neural control of human locomotion with particular focus on understanding the role of morphology. Although several investigations of bipedal locomotion have been undertaken in the fields of biology, robotics and adaptive behavior which address the issues of physical dynamics or neural control independently, few have focused on uncovering the relationships between these two aspects. The collection of work in this thesis addresses this issue. The thesis is a compilation of publications, which are divided into three categories. The first category is the investigation of the interaction between morphology and neural control using abstract simulation models and theory. The models use implementations of morphology as well as neural control which are simplified from the biological situation, in order to eliminate complexity and focus on underlying principles. The second category is the study of the physical dynamics of the biped morphology through the development of real robots. Previously, it has been shown that the lower body can have passive dynamics which assist the control of locomotion. However, the effect of upper body dynamics has not been extensively investigated. The development of the STUMPY hopping robot and the BENDY walking robot serve to illuminate the role of upper body dynamics in the control of locomotion. The third category focuses on investigating the interactions between the known neural components of the human spinal cord with the dynamics of the body. One method used is the development of a human neuro-musculo-skeletal model that closely mimics the topology of spinal pathways and the biomechanics of the human lower body. Another method is the use of a partial model of spinal sensorimotor pathways, to investigate the role of sensory modalities in gait generation. The results of these studies demonstrate that the dynamics of the body play an important role in the neural control of locomotion and suggest that they may be exploited during human walking to reduce complexity of neural control. The new insights represent not only a scientific contribution, but are also relevant for the rehabilitation of spinal cord injured patients.
i
Acknowledgments I’d like to thank my advisor, Prof. Rolf Pfeifer, for giving me the opportunity to work in one of the most exciting fields of science, to explore new ideas and to eventually chart my own course in the field. In the course of my work, I have also had the opportunity to work with some excellent colleagues. In particular I would like to thank Dr. Josh Bongard, Raja Dravid, Fumiya Iida, Kojiro Matsushita, Mario Bellotti, Dr. Saso Jezernik and Dr. Armin Curt for their collaboration and support. I’d also like to thank my students Andreas Durrer, Tobias Kueng and Daniel Baumgartner for helping to lay the foundations of this project. A very special thanks go to Prof. Hiroshi Yokoi for not only providing me with creative input and stimulus at various stages of my work, but also for helping me develop my practical skills in robotics, which I may not have had the opportunity to develop otherwise. As always, healthy criticism is key to producing high quality research and I have had the opportunity of receiving some very constructive criticism during my work from Dr. Andre Seyfarth, Hartmut Geyer and Prof. Henrik Hautop Lund. I am grateful to them for their time and effort. Then of course, there are the friends who in both small and big ways helped me get through the day-to-day trials of doing a PhD. Particularly, I would like to thank Dr. Christian Ridderstrom, Carmela Ahokas, Dr. Shonali Pachauri and Dr. Matt Hare for being there over the years as well as my friends Laura Tannenbaum, Minni Shahi and Christy Palangattil from home. Its been a rough road at times, and I might have ended up very troubled indeed without friends to air out my concerns. Thank you all for putting up with me! (I know it hasn’t always been easy..) I’d also like to acknowledge my colleagues at the AI Lab who have provided their indelible advice and companionship, in particular Dr. Verena Hafner, David Andel, Miriam Fend, Max Lungarella, Lukas Lichtensteiger, Dale Thomas and other members of our lab. Thank you all for being there and for making sense! I’d also like to thank my family for, well, being my family. My father, my mother, my sister Tias and my brother Vivek are like pillars of my constitution. It would have been difficult without them. Finally, I’d like to thank Switzerland for giving me the opportunity to live in the heart of Europe and have the experience of a lifetime, traveling to all corners of the Continent, snowboarding in the Alps, seeing some of the most picturesque views in the whole world and, of course, having an endless supply of Swiss chocolate. Merci vielmal! – Chandana Paul
ii
Contents 1 Introduction 1.1 Background . . . . . . . . . . . . . . . . . . . 1.2 Neural control of Biped Locomotion: A review 1.2.1 Natural system . . . . . . . . . . . . . 1.2.2 Artificial systems . . . . . . . . . . . . 1.3 Motivation . . . . . . . . . . . . . . . . . . . . 1.4 Research Methods . . . . . . . . . . . . . . . . 1.5 Outline of the Thesis . . . . . . . . . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
1 1 2 2 5 9 10 11
Part I
14
2 Effect of Morphological Change 2.1 Introduction . . . . . . . . . . 2.2 The Robot . . . . . . . . . . . 2.3 Neural Controller . . . . . . . 2.4 The Genetic Algorithm . . . . 2.5 Experiments . . . . . . . . . . 2.6 Results . . . . . . . . . . . . . 2.6.1 Microscopic Changes . 2.6.2 Mid-Range Changes . 2.6.3 Macroscopic Changes 2.7 Discussion . . . . . . . . . . . 2.8 Conclusions . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
15 15 18 19 20 21 22 22 22 22 23 27
3 Neural Coupling through Morphology 3.1 Introduction . . . . . . . . . . . . . . . . . . . . 3.2 The Robot . . . . . . . . . . . . . . . . . . . . . 3.3 Neural Controller . . . . . . . . . . . . . . . . . 3.3.1 Complete Bilateral Decoupling . . . . . . 3.3.2 Bilateral Decoupling with Global Sensing 3.3.3 Initial Condition . . . . . . . . . . . . . 3.4 The Genetic Algorithm . . . . . . . . . . . . . . 3.5 Results . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Complete Bilateral Coupling . . . . . . . 3.5.2 Bilateral Decoupling with Global Sensing
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
28 28 30 31 31 33 34 34 35 35 38
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
iii
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
3.6 3.7
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4 Computational Role of Morphology 4.1 Introduction . . . . . . . . . . . . . . . . . . . . 4.2 The XOR Robot: a thought experiment . . . . . . . . . . . . . . . 4.3 The OR Robot . . . . . . . . . . . . . . . . . . . 4.4 Explicit Computation . . . . . . . . . . . . . . . 4.5 Complex Morphology . . . . . . . . . . . . . . . 4.6 Discussion . . . . . . . . . . . . . . . . . . . . . 4.6.1 Morphological Computation . . . . . . . 4.6.2 Explicit morphological computation . . . 4.6.3 Using computational and motor functions 4.6.4 Duality . . . . . . . . . . . . . . . . . . 4.6.5 Effect of the environment . . . . . . . . . 4.6.6 Morphology and Control . . . . . . . . . 4.7 Conclusion . . . . . . . . . . . . . . . . . . . .
43 . . . . . . . . . . . . . . 44 . . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
44 45 47 49 51 51 51 51 51 52 52 53
Part II
55
5 The STUMPY Hopping Robot 5.1 Introduction . . . . . . . . . . . . . 5.2 Robot Mechanical Structure . . . . 5.3 Modelling and Analysis . . . . . . . 5.4 Control . . . . . . . . . . . . . . . 5.4.1 Tuning the Hopping Height 5.4.2 Straight Walking . . . . . . 5.4.3 Reversing Direction . . . . 5.4.4 Control of Turning Rate . . 5.5 Discussion . . . . . . . . . . . . . . 5.6 Conclusions . . . . . . . . . . . . .
. . . . . . . . . .
56 56 58 59 61 62 62 62 62 64 65
. . . . . . . . . . .
67 67 69 70 70 71 72 73 73 75 75 76
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
6 The BENDY Humanoid Robot 6.1 Introduction . . . . . . . . . . . . . . . . . . . . 6.2 Robot Mechanical Design . . . . . . . . . . . . . 6.2.1 Trunk Balance Mechanism . . . . . . . . 6.2.2 Cable Tendon Mechanism . . . . . . . . 6.2.3 Passive Swing . . . . . . . . . . . . . . 6.2.4 Shoulder Mass Spring System . . . . . . 6.3 Control . . . . . . . . . . . . . . . . . . . . . . 6.4 Experimental Results . . . . . . . . . . . . . . . 6.5 Discussion . . . . . . . . . . . . . . . . . . . . . 6.5.1 Overall Performance . . . . . . . . . . . 6.5.2 Relationship to frequency and amplitude iv
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
6.6
6.5.3 Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.5.4 Role of the upper body . . . . . . . . . . . . . . . . . . . . . . . . 77 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Part III
79
7 Neuro-Musculo-Skeletal Model 7.1 Introduction . . . . . . . . . . . . . . . . . . 7.2 The neuro-musculo-skeletal model . . . . . . 7.2.1 Neural Spinal Cord Model . . . . . . 7.2.2 Musculo-skeletal Model . . . . . . . 7.2.3 Foot-ground reaction model: . . . . . 7.2.4 Lokomat Treadmill Conditions: . . . 7.3 Subcomponent Validation . . . . . . . . . . . 7.3.1 CPG Tests . . . . . . . . . . . . . . . 7.3.2 Muscle Spindle Tests . . . . . . . . . 7.3.3 Golgi Tendon Organ Tests . . . . . . 7.3.4 Cutaneous Pathway Tests . . . . . . . 7.4 Overall Model Behavior . . . . . . . . . . . 7.5 Experiments and Clinical Relevance . . . . . 7.5.1 Experiments . . . . . . . . . . . . . 7.6 Discussion . . . . . . . . . . . . . . . . . . . 7.6.1 Topological vs. Functional Modelling 7.6.2 Clinical Application . . . . . . . . . 7.7 Conclusion . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
80 80 83 83 87 91 92 93 93 93 94 95 96 97 98 103 105 106 107
8 Sensorimotor Control 8.1 Introduction . . . . . . . . . . 8.2 Biped Morphology . . . . . . 8.3 Sensorimotor Neural Network 8.3.1 Initial Condition . . . 8.4 The Genetic Algorithm . . . . 8.5 Results . . . . . . . . . . . . . 8.6 Discussion . . . . . . . . . . . 8.7 Conclusion . . . . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
108 108 110 111 113 113 114 117 122
. . . . . . . .
124 124 126 128 129 130 131 131 133
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
9 Discussion 9.1 Summary of Research . . . . . . . . . . . . 9.2 Role of the Body in Neural Control . . . . . 9.3 Material Properties . . . . . . . . . . . . . 9.4 Role of the Upper Body . . . . . . . . . . . 9.5 Proprioceptive vs. foot contact information 9.6 New model of spinal neural architecture . . 9.7 Coordination of CPG and Reflexes . . . . . 9.8 Products . . . . . . . . . . . . . . . . . . . v
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
9.8.1 Neuro-musculo-skeletal model . . . . . . 9.8.2 STUMPY . . . . . . . . . . . . . . . . . 9.8.3 BENDY . . . . . . . . . . . . . . . . . . 9.9 Personal Experience . . . . . . . . . . . . . . . . 9.10 Implications for Rehabilitation after Spinal Injury 9.11 Summary of Contributions . . . . . . . . . . . .
vi
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
133 135 135 136 138 140
List of Figures 1-1 1-2 1-3 1-4
Lokomat Rehabilitation Therapy at Balgrist University Hospital . . . . . Organization of the motor system (Kandel, 1991) . . . . . . . . . . . . . (a) BIPER-3 (b) WABIAN - RLII (c) Honda P2 . . . . . . . . . . . . . . Robots which include passive dynamics. (a) Passive dynamic walker by McGeer (top left) (b) BAPS (top right) (c) MIKE (bottom left) (d) Spring Flamingo (bottom right) . . . . . . . . . . . . . . . . . . . . . . . . . .
. . .
1 3 5
.
7
2-1 Biped construction: Fig. a) shows the biped skeleton and its 6 degrees of freedom Fig. b) shows the biped with attached mass blocks. (T and P refer to tactile and proprioceptive sensors respectively, and M indicates a motor.) 2-2 Pictorial representation of the neural network used to control both types of agents. T1 and T2 correspond to the two touch sensors, P1 through P6 indicate the six proprioceptive sensors, and M1 through M6 indicate the six torsional motors of the biped. B1 and B2 indicate the two bias neurons included in the network. . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3 (a) Best fitness achieved in each generation with microscopic mass distribution changes (b) Morphology of most fit biped achieved with microscopic mass distribution changes . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4 (a) Best fitness achieved in each generation with mid-range mass distribution changes (b) Morphology of most fit biped achieved with mid-range mass distribution changes . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5 (a) Best fitness achieved in each generation with macroscopic mass distribution changes (b) Morphology of most fit biped achieved with macroscopic mass distribution changes . . . . . . . . . . . . . . . . . . . . . . . 2-6 Changes in vertical position of biped Center of Mass: The thin line tracks the changes of the vertical CoM for the most fit biped in the microscopic case. The middle line tracks the CoM of the best biped in the mid-range case and the thick line is that of the best biped in the macroscopic case. . . 2-7 Trajectories of CoM of three Bipeds: The light gray trajectory shows the performance of an evolved biped. The dark gray line shows the performance of one of its children, which has seven control and one morphological mutation. The black trajectory shows the performance of a third biped, which is equivalent to the more fit child, except that the morphological mutation is suppressed. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vii
18
19
22
23
24
25
26
3-1 The biped physical structure and its six degrees of freedom. The labels indicate the joint angle and foot contact sensors. The widths of the biped links vary under evolutionary control within the range shown in Table 1. . 3-2 Bilaterally decoupled neural controller including only joint angle and foot contact sensors. The input nodes for the left leg controller are LR: Left Hip Roll, LP: Left Hip Pitch, LK: Left Knee Pitch, LT: Left foot contact and the outputs are to the left hip roll, hip pitch and knee joints. For the right leg controller the inputs are RR: Right Hip Roll, RP: Right Hip Pitch, RK: Right Knee Pitch, RT: Right Foot Contact and the outputs are to the right hip roll, hip pitch and knee joints. B and B are bias nodes. . . . . . . . 3-3 Bilaterally decoupled neural controller enhanced with global sensing. In addition to the nodes in the bilaterally decoupled controller in Figure 3-2, the left leg controller has sensory input nodes for WO: waist orientation in transverse plane, WP: waist sagittal position and FP: left foot sagittal position. For the right leg controller the input to FP is the foot position of the right foot instead. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4 Best fitness in each generation, for the six evolutionary optimization experiments with complete bilateral decoupling. . . . . . . . . . . . . . . . . . 3-5 Trajectories of the most successful bipeds in each of the six experiments. (Some of the trajectories overlapped.) . . . . . . . . . . . . . . . . . . . 3-6 Physical Structure of the biped which achieved the highest fitness with complete bilateral decoupling. . . . . . . . . . . . . . . . . . . . . . . . 3-7 Foot placement history of the biped with highest fitness. Each footprint is represented by a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8 Footsteps of the biped with highest fitness. Dark lines indicate that the foot is in contact with the ground. The length of the line indicates the duration of time for which the foot was in contact with the ground in one step. . . . 3-9 Steady state phase plots of the joint angles of the biped with highest fitness. The x-axis plots and the y-axis . . . . . . . . . . . . . . . . . . . . . . 3-10 Motor Neuron Activations of the Right Hip Pitch, Hip Roll and Knee joints, of the biped with highest fitness. . . . . . . . . . . . . . . . . . . . . . . 3-11 Best fitness in each generation, for the six evolutionary optimization experiments on bilateral decoupling with global sensing. . . . . . . . . . . . . 3-12 Trajectories of the most successful bipeds in each of the six experiments. (Some of the trajectories overlapped.) . . . . . . . . . . . . . . . . . . . 3-13 Physical Structure of the biped which achieved the highest fitness with bilateral decoupling and global sensing. . . . . . . . . . . . . . . . . . . . 3-14 Foot placement history of the biped with the highest fitness. Each footprint is represented by a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-15 Footsteps of the biped with the highest fitness. Dark lines indicate that the foot is in contact with the ground. The length of the line indicates the duration of time for which the foot was in contact with the ground in one step. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16 Steady state phase plots of the joint angles of the biped with highest fitness. The x-axis plots and the y-axis . . . . . . . . . . . . . . . . . . . . . viii
. 31
. 32
. 33 . 35 . 35 . 36 . 36
. 37 . 37 . 38 . 38 . 39 . 39 . 40
. 40 . 41
3-17 Motor Neuron Activations of the Right Hip Pitch, Hip Roll and Knee joints of the best biped with highest fitness. . . . . . . . . . . . . . . . . . . . . . 42 4-1 The XOR Robot: This robot has one wheel, with two actuated degrees of freedom. The motor M is responsible for turning the wheel so that the robot moves forward. The motor M is responsible for lifting the wheel off the ground. Each motor is controlled by a separate perceptron network, which takes as inputs A and B. M is controlled by a network which computes A OR B, and M by a network which computes A AND B. Using only these controllers, the robot is able to display the XOR function in its behavior. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2 Computational structure equivalent to the XOR Robot: The body of the XOR robot acts as if it is performing the computational function M AND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3 The OR Robot: This robot has two wheels both of which turn in the same direction when actuated. Each wheel is actuated by one motor which is responsible for turning the wheel. The wheels are also capable of being driven passively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-4 Conventional neural network architecture for Vaccuum Cleaning Robot: The robot is controlled by a neural network which has an input layer with sensory input nodes, labelled S, input node I and bias node B. There are two hidden layers, each with 4 hidden neurons each, labelled H. The final layer is the output layer which has two output nodes to send motor commands to the robot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5 Morphological computation used in the neural control of the Vaccuum Cleaning Robot: The robot is controlled by a neural network which has an input layer with sensory input nodes, labelled S, input node I and bias node B. There are two hidden layers. The first hidden layer has 2 hidden nodes, represented by H, and two nodes which convey motor commands to the robot, represented by M and M . The second hidden layer also has two hidden nodes, labelled H, and a node which conveys the output of the robot accelerometer A. The final layer is the output layer which has two output nodes which send motor commands to the robot. . . . . . . . . . . 4-6 Control of a robot manipulator: A 2 DOF manipulator is is controlled by a neural network with sensory feedback. The network has an input layer with 2 sensory input nodes, labelled S, an input node I, and a bias node B. The sensory nodes recieve proprioceptive inputs from the manipulator on joint angles. There is one hidden layer with four hidden nodes represented by H. The output layer has two output nodes which send motor commands to the two actuators of the robot, M and M . . . . . . . . . . . . . . . .
. 45
. 46
. 46
. 47
. 48
. 50
5-1 STUMPY Robot: photograph of the robot Fig. . . . . . . . . . . . . . . . . 58 5-2 Schematic diagram of the STUMPY Robot, with variables which are used in modelling and analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . 59
. . . . . . . . . . . . . 63 5-3 Forward walking, produced when ix
. . . . . . . . . . Backward walking, produced when Turning left, with a small turning rate, produced when ! " . . . . . . Turning right, with a small turning rate, produced when ! # " . . . . . Going backwards and turning right, with a small turning rate, produced # " . Simply by changing to % , the robot walks backward, when $ as it slowly turns right. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-8 Going backwards and turning left, with a small turning rate, produced when ! " . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-9 Turning right, with a large turning rate, produced when #&# . With this controller the robot can effectively turn in place. . . . . . . . . . . . & . Again 5-10 Turning left, with a large turning rate, produced when ' the robot can effectively turn in place . . . . . . . . . . . . . . . . . . . .
5-4 5-5 5-6 5-7
6-1 The BENDY robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2 Degrees of freedom of the robot. P indicates a passive pitch joint, and R, a passive roll joint. The zig-zag lines in the upper body indicate springs. . . 6-3 Leg construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-4 Waist Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5 Cable Tendon Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . 6-6 The H8 3664F microcontroller board . . . . . . . . . . . . . . . . . . . . 6-7 Trajectory of the robot walking with oscillation amplitude of (*) and frequency of 0.71 Hz. The trajectory has been obtained by tracking an LED attached at the robot’s waist with an overhead camera. . . . . . . . . . . . 6-8 Data from foot contact sensors plotted with respect to upper body inclination. The top graph shows the upper body position with a maximum value representing inclination to the left and minimum value, inclination to the right. The middle graph plots the activation of the four foot contact sensors on the left foot, and the bottom graph plots the activation of the four foot contact sensors on the right foot. . . . . . . . . . . . . . . . . . . . . . . 6-9 A plot of the average speed acheived by the robot under varying conditions of amplitude and frequency. . . . . . . . . . . . . . . . . . . . . . . . . .
. 63 . 64 . 64
. 65 . 65 . 66 . 66 . 68 . . . . .
. 74
. 75 . 76
7-1 Diagram indicating the overall structure of the neuro-musculo-skeletal model. 7-2 (a) Central Pattern Generator circuit for a single joint (A + represents an excitatory connection and represents an inhibitory connection) (b) Phase relationships between hip, knee and ankle oscillators (A + represents an excitatory connection and represents an inhibitory connection) . . . . . . 7-3 (a) Spinal reflex pathways corresponding to muscle spindle and Golgi tendon organ receptors ( + represents an excitatory connection and represents an inhibitory connection) (b) Reflex pathways corresponding to right and left cutaneous sensors ( + represents an excitatory connection and represents an inhibitory connection . . . . . . . . . . . . . . . . . . . . . . 7-4 Definition of the variables of the skeletal model. . . . . . . . . . . . . . . .
x
70 71 71 72 73
82
85
86 88
7-5 Results of validation test on the Central Pattern Generator: (a) Motoneuron activation (MN) in the muscles of the hip, knee and ankle joints (The solid line represents the extensor and the dotted line, the flexor.) (b) Joint angle displacements (q) of the right and left hip, knee and ankle joints . . . . . . 7-6 Results of validation test on the muscle spindle related pathways of the knee joint: (a) muscle length (top left) and muscle velocity (bottom left) (b) motor neuron activation (top middle) and muscle spindle receptor activation (bottom middle) (c) joint torque (top right) and joint angle (bottom right) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-7 Results of validation tests on the Golgi tendon organ related pathways of the knee extensor: (a) Force of perturbation applied to evoke the reflex response (top) (b) motor neuron activation of the muscle (second from top) (c) muscle activation (third from top) (d) Golgi sensor activation (bottom) . 7-8 Results of validation tests on the Golgi tendon organ related pathways of the knee extensor: (a) Force of perturbation applied to evoke the reflex response (top) (b) motor neuron activation of the muscle (second from top) (c) muscle activation (third from top) (d) Golgi sensor activation (bottom) . 7-9 Results of validation test of the cutaneous pathways, when stimulation is applied to the left foot: (a) activation of extensor muscles of the left leg (first column from left) (b) light activation of the flexor muscles of the right leg (second column) (c) Joint torques of the left leg (third column) (d) Joint torques of the right leg (fourth column) . . . . . . . . . . . . . . . . 7-10 Joint angle trajectories when the biped has acheived a stable limit cycle, plotted for 4s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-11 Phase plots of joint angle vs. angular velocity, plotted for 10s. . . . . . . . 7-12 Gait pattern after the biped has reached a stable limit cycle, shown for 2s . . 7-13 Movement pattern achieved in the control condition [ ,*-/. 0 12 , ,*3 4 12 , ,*5 4 12 ] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-14 Movement pattern acheived in the case [ ,6-/. 1 ) , ,*3 4 7 12 , ,*5 4 18 ] 7-15 Movement pattern acheived in the case [ ,6-/. 12 , ,*3 4 7 1 ) , ,*5 4 18 ] 7-16 Movement pattern achieved in the case [ ,6-/. 12 , ,*3 4 7 12 , ,*5 4 1 ) ]
94
95
96
97
98 100 102 102 103 103 104 104
8-1 The biped morphology: The lower body biped is a 7-link structure with 8 degrees of freedom. Each hip, knee and ankle joint has one degree of freedom in the sagittal plane. Additionally each hip joint has one degree of freedom in the frontal plane. . . . . . . . . . . . . . . . . . . . . . . . . . 110
xi
8-2 The sensorimotor neural network controller: This is a bilaterally decoupled controller, each half of which only has direct connections between the sensors and the motors for each leg. The sensors of the input layer are labeled as follows. WO: waist orientation WLF: difference between waist sagittal position and left foot sagittal position WH: waist height LR: left roll LP: left pitch LK: left knee LT: left foot contact (or touch sensor) RK: right knee joint angle position RP: right hip pitch joint angle position RR: right roll joint angle position RT: right foot contact (or touch sensor) WRF: difference between waist sagittal position and right foot sagittal position. The output layer consists of the motoneurons for the hip pitch (HP), hip roll (HR), knee (K) and ankle (A) joints. . . . . . . . . . . . . . . . . . . . . 8-3 Fitness Graphs: The graphs plot the highest fitness achieved in each generation, for each experiment. H denotes experiments conducted with the healthy sensorimotor network (top left). L1 denotes a lesion of the cutaneous sensors (top right). L2 denotes a lesion of the joint angle sensors (middle left). L3 denotes a lesion of the waist height sensor (middle right). L4 denotes a lesion of the waist orientation sensor (bottom left) L5 denotes a lesion of the waist-to-foot sagittal position sensor (bottom right). . . . . 8-4 Average Fitness: Each of the curves represents the average of the highest fitness in each generation over 20 experiments. Graphs are shown for experimental conditions, H (solid line) and L1-L5 (dotted lines). . . . . . 8-5 Cutaneous Network: This is essentially the sensorimotor network with all sensory inputs, excluding cutaneous information, removed. The sensors of the input layer are LT: left foot contact (or touch sensor) and B1: bias node and RT: right foot contact (or touch sensor) and B1: bias node. As before, the output layer consists of the motoneurons for the hip pitch (HP), hip roll (HR), knee (K) and ankle (A) joints. . . . . . . . . . . . . . . . . . . . . 8-6 Fitness History (CUT): The graphs indicate the highest fitness in each generation, for each of the 20 experiments with the cutaneous network . . . . 8-7 A short movement sequence depicting one step (CUT): In the top four images the left leg steps forward, and in the bottom four images, the right leg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-8 Evolved synaptic weights of the best walker (CUT) . . . . . . . . . . . . 8-9 Motoneuron Outputs (CUT): The graphs indicate the motoneuron outputs of the right hip pitch (top) hip roll (second from top), knee (third from top) and ankle joint (fourth from top) for 1000 time steps. . . . . . . . . . . . 8-10 Joint angle positions over time (CUT): The graphs track the joint angle positions for the joints of the left and right legs over 1000 time steps . . . 8-11 Joint angle phase plots (CUT): The graphs depict phase plots of joint angle vs. velocity over 1000 time steps. . . . . . . . . . . . . . . . . . . . . . .
. 112
. 115
. 116
. 117 . 118
. 119 . 119
. 120 . 121 . 122
9-1 Effect of mass distribution on neural control through the intersegmental dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
xii
9-2 NSCM software tool for clinical studies: (a) NNetview graphical neural network design tool (b) network represented as text file (c) Biomechanical body model controlled by neural network, running under Matlab (d) Movement data stored in Matlab arrays . . . . . . . . . . . . . . . . . . . 134 9-3 STUMPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 9-4 BENDY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
xiii
List of Tables 2.1
Morphological parameters of the agent. A ul represents one unit length defined as the radius of the spherical sockets at the hip and knees. A um represents one unit mass defined as the mass of the same spherical socket. The parameters in boldface vary under evolutionary control between the ranges shown below. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1
Morphological parameters of the biped robot. A ul represents one unit length defined as the five times the radius of the spherical sockets at the hip and knees. A um represents one unit mass defined as the mass of the same spherical socket. The ranges of the parameters which vary under evolutionary control are shown in square brackets. . . . . . . . . . . . . . . 29
6.1
Results: This table shows the distance travelled by the robot in 20 seconds, under varying conditions of amplitude and frequency of oscillation. Distances are shown in cm. . . . . . . . . . . . . . . . . . . . . . . . . . . 74
7.1
Anthropomorphic Parameters of the Musculo-skeletal model have been set according to Winter, 1990. . . . . . . . . . . . . . . . . . . . . . . . . . Maximal Muscle Forces . . . . . . . . . . . . . . . . . . . . . . . . . . Values of the constants of the passive muscular model part as identified for a healthy subject (Riener 1999) . . . . . . . . . . . . . . . . . . . . . . . Experimental Conditions in Deactivation Study: In condition 1 the muscle spindle related reflex pathways are deactivated by setting the gain to 0, in condition 2 the golgi tendon organ related pathways are similarly deactivated, and in condition 3 the cutaneous reflex pathways gains. . . . . . . Results of Deactivation Studies . . . . . . . . . . . . . . . . . . . . . . . Experimental Conditions in Gain Modulation Study: In Set 1 muscle spindle related reflex pathway gains are altered, in Set 2 the golgi tendon organ related pathway gains, and in Set 3 the cutaneous reflex pathway gains. The line in bold indicates the control condition, which is identical over all sets. Results of Gain Modulation Studies . . . . . . . . . . . . . . . . . . . .
7.2 7.3 7.4
7.5 7.6
7.7
xiv
. 87 . 90 . 91
. 99 . 99
. 101 . 101
8.1
Morphological parameters of the biped robot. A ul represents one unit length defined as the five times the radius of the spherical sockets at the hip and knees. A um represents one unit mass defined as the mass of the same spherical socket. The ranges of the parameters which vary under evolutionary control are shown in square brackets. . . . . . . . . . . . . . . 111
xv
Chapter 1 Introduction The goal of this project was to investigate the neural control of human locomotion, with particular focus on understanding the role of morphology. Although human neurophysiology has been studied to a large extent as well as biomechanics, there have been very few studies which bridge the gap between these two fields and attempt to uncover the relationships between the physical characteristics of the human musculo-skeletal system, broadly described as morphology, and the neural control. The work in this thesis focused on this issue.
1.1
Background
The conception of this project was in a clinical context. Spinal cord injury can be a debilitating event often leading to severe loss of motor functionality. Depending on the location and severity of the injury, a patient can suffer from partial motor dysfunctions to paraplegia or even quadruplegia. The prognosis for recovery is mixed; in some cases with appropriate therapy and care a patient is able to recover some or all of their motor functions, but in other cases the losses are permanant.
Figure 1-1: Lokomat Rehabilitation Therapy at Balgrist University Hospital Rehabilitation of spinal cord injured patients is still only a partially solved problem. It is 1
known that treadmill therapy, in which patients are suspended on a treadmill and manually assisted by a therapist in moving their legs, has proved beneficial in several cases [20]. Similarly, therapy using a Driven Gait Orthosis (Fig. 1-1), a robotic exoskeleton which helps the patients legs move through a simulated gait cycle also seems beneficial [16] [50]. However, as the basic understanding of the neural control of human locomotion is poor, fundamentally it is not understood why these therapy techniques work. Therefore, it is difficult to know how to improve them. In the interests of better understanding the control of locomotion, an interdisciplinary research project was established between the Balgrist Paraplegic Center, the AI Lab at the University of Zurich, and the ETH Automatic Control Group. The main goal was to study the neural control of walking at the spinal level. The Balgrist Paraplegic Center would perform clinical studies with patients, the AI Lab would use synthetic methodology, involving simulations and robots, to study the control of walking and the Automatic Control Group would model the biomechanics of the human musculo-skeletal system. It was hoped that this highly interdisciplinary collaborative effort would lead to a better understanding of the complex system of human locomotor control.
1.2 1.2.1
Neural control of Biped Locomotion: A review Natural system
Organization of the motor CNS The areas of the central nervous system which are involved in motor control in humans have been mapped to a large extent, and are now basic knowledge in neuroscience. Here, Kandel et al. [53], will be used as the authoritative source on the subject. It is known that the central nervous system has three main layers involved in motor control: the motor areas of the cerebral cortex, the descending systems of the brain stem and the spinal cord. These systems are organized both hierarchically and in parallel, as illustrated in Figure 1-2. The hierarchical organization enables the higher centers to give general commands, which can then be more specifically implemented in the lower levels. The parallel organization enables the higher centers to send commands directly to the lowest level. Thus, each of the higher brain centers has direct connections to motor neurons in the spinal cord. The parallel organization means that if one of the tracts is severely impaired, another can take over its function to some extent. The spinal cord is the lowest neural layer in the organization of motor control. The main role of the spinal cord is to produce fast stereotyped responses to sensory inputs (reflexes) and simple rhythmic oscillations. The brain stem is the next level in the organization. It serves to integrate visual and vestibular information with somatosensory inputs and regulate the activation of the spinal circuits, mainly for the control of posture. The motor areas of the cerebral cortex form the highest layer of organization, and are mainly responsible for planning complex sequences of movements. In addition to these three hierarchically organized areas, the cerebellum and the basal ganglia are also involved in the control of movement. The cerebellum is responsible for comparing the desired movements to the sensory information obtained from the spinal 2
Cerebral cortex Motor areas Thalamus Basal ganglia Cerebellum
Brain stem
Spinal cord
Muscle contraction and movement
Sensory receptors
Sensory consequences of movement
Figure 1-2: Organization of the motor system (Kandel, 1991) cord and send correcting signals to the spinal cord via the brain stem. The basal ganglia supports the cerebral cortex in motor planning by acquiring associations between tasks that are frequently performed in succession. Due to this organizational structure, the spinal cord directly or indirectly receives inputs from all the different brain areas involved. The cortico-spinal tracts which are significant in size convey information directly from the cortex to the spinal cord. The tracts descending from the brain stem are also significant in that they carry information both from the brain stem and the cerebellum. Spinal reflexes The spinal cord has several reflexes. A reflex is a connection from a sensory input neuron to a motoneuron, which may be direct or include intermediate interneurons. Activation of the sensory neuron leads to a rapid activation or deactivation of the associated motoneuron. The majority of reflexes lie in the spinal cord, although a few, involving the control of posture, also go via the brain stem. This is because reflexes must be fast responses to critical external situations and require short neural latencies, which is usually not possible if a signal must travel the entire length of the spinal cord up to the brain. There are several categories of spinal reflexes. Some relate to maintaining the functionality of individual muscles. Others relate to coordinating the activity of pairs of muscles, and yet others relate to coordinating whole groups of muscles in complex motor activities. There are several sensory modalities involved in reflexes. The muscles contain two differ3
ent kinds of sensors: muscle spindles which sense muscle length and velocity, and golgi tendon organs which sense muscle force. The muscle spindles are involved in the stretch reflex, which ensures that each muscle maintains appropriate muscle tension. They are also involved in the crossed inhibition reflex which ensures that antagonist muscles do not have high simultaneous activation. The golgi tendon organ is involved in a positive force feedback loop which supports activities such as walking [91] and running [31], and also in a negative feedback loop which ensures that if the muscle force becomes dangerously high, the muscle is deactivated preventing it from physical damage. In addition to these sensory modalities, there are also receptors at the joints which measure joint angles, and contact sensors on the feet. The joint angle sensors, particularly at the hips are believed to be involved in reflexes which coordinate groups of leg muscles to regulate stance and swing phase during locomotion. The cutaneous sensors are also involved in reflexes which coordinate groups of muscles to enhance the control of stance phase. They are also known to be involved in reflexes which coordinate several muscles of a leg to rapidly withdraw from painful stimuli. Spinal pattern generators A spinal pattern generator is a variation of a spinal pathway in which two interneurons are reciprocally interconnected by inhibitory synapses. This leads to a neural unit which is capable of generating rhythmic oscillation, even in the absence of sensory or central input. These simple spinal units, also called central pattern generators (CPGs), are believed to be used in the control of various rhythmic behaviors, such as scratching, crawling and walking. The most prevalent architecture for the connection of a neural oscillator to the musculoskeletal system, is the half-center model proposed by Brown [10]. In this architecture, each of the two reciprocally connected neurons, also called half-centers, are connected to antagonist muscles. As the half-centers are alternately activated, the connection leads to alternate activation of the flexor and extensor muscles, leading to rhythmic movement. The CPGs also receive sensory inputs. When connected to sensory inputs, the output frequency of the CPG can entrain to the frequency of the sensory information so that the overall system reaches a stable limit cycle. The CPGs can also be connected to each other, in which case they also entrain to each other to converge to a global limit cycle. Such central pattern generators have been shown to exist in the spinal cord of several vertebrate species [33]. In lamprey it has been shown that the oscillatory motions during swimming are caused by segmental oscillators which are interconnected such that waves of movement occur [34]. In salamander it has been shown that such oscillators can support both aquatic and terestrial movements [49]. In cats it has been shown that even in the complete absence of supraspinal control, when placed on a treadmill they can initiate rhythmic walking movements using spinal oscillators [35]. It is believed that such CPGs also exist in the human spinal cord, supporting rhythmic activities like crawling, walking and running. It is known that in the control of locomotion, all the subsystems described above are involved to a large extent. However, much remains to be understood about how they coordinate with each other to produce overall behavior.
4
Figure 1-3: (a) BIPER-3 (b) WABIAN - RLII (c) Honda P2
1.2.2
Artificial systems
Artificial systems such as simulations and robots can serve as good models for understanding the principles of locomotion. There have been several artificial models of biped locomotion. The first actuated biped walking machines were built at Waseda University. WL 5, a hydraulically powered biped walking machine was built in 1972 by Kato et al. [55]. This biped walked statically by controlling its joints to track prespecified joint angle trajectories and acheived a speed of 10cm/min. In 1977, Hemami et al. [43] showed the first biped walking in simulation. This was a 2D model which used human joint angle data to create reference trajectories, which set the desired joint angle values for the robot at each instant in time. In 1981 Kato et al. [56], improved on their earlier work to develop a quasi-dynamic biped walking machine. This machine walked at a speed of approximately 3 m/min. In 1984 Miura and Shimoyama designed BIPER-3 and BIPER-4, the first bipeds which could walk dynamically by generating reference trajectories based on an inverted pendulum model with linearized equations of motion [73]. Although successful, all these early models used pre-specified joint reference trajectories, and thus, displayed behavior which was quite far from the highly adaptive nature of human locomotion. Thus, it was unlikely that such control architectures portrayed accurate models of the control of human locomotion. The first adaptive controller for biped walking was implemented by Su and Zheng in 1989 [110]. In their work, a multilayer feed-forward neural network with one hidden layer was used, which was trained using a version of back-propagation learning, to produce appropriate joint torque outputs. The training was based on sensor inputs from joint angle and foot contact sensors. Using this technique, the SD-2 robot developed by Zheng et al. was shown to be able to walk on flat ground and varying slopes [123]. Although it used neural adaptation, the gait was still pre-generated off-line and then applied to the biped. In 1991 Taga developed a model in which each joint was controlled by one neural oscillator which consisted of two reciprocally inhibited neurons [111]. As the model had seven degrees of freedom, one at each hip, knee and ankle, and a torso joint, the pattern generator 5
consisted of seven neural oscillators. The walking gait pattern was divided into six phases, and the output of the neural oscillators were gated according to the current phase. The oscillators were also integrated with sensory information. The angles of the joints in the earth fixed reference frame are passed to the oscillators, in a manner such that extreme movement in one direction caused the oscillator to output commands to move in the opposite direction. The sensory information was also modulated by the global state, so that the angle information was only sent to the oscillator during certain states. This mechanism, enabled the biped simulation to walk and adapt to small changes in environmental conditions. In 1992, Lee et al. [60] used neural networks for the synthesis of gait for a simplified biped model constrained to the sagittal plane, without knees, massless legs, and point contacts as feet. The neural network was trained to generate hip actuator torques using a backpropagation based supervised learning algorithm. The training data was produced by three linear controllers, which calculated the hip actuator torque based on initial disturbance, average walking speed, and stride length. After training on over 7000 training patterns, the network was able to generate a torque trajectory which represented the combined results of these three different controllers and maintain a walking gait. In 1996, Kun and Miller used three Cerebellar Model Articulation Controller (CMAC) neural networks to design an adaptive controller for a 10 degree-of-freedom (DOF) biped robot with upper body [58]. The first network was used to control the distribution of force between the heels and the toes for balance. The second was used to predict the amplitude and the velocity of the robots side-to-side motion, in order to control lateral balance. The third was used to predict ankle positions which would lead to kinematically consistent robot postures. Using these components, the robot was able to display a fair amount of adaptivity in its gait. In 1997, Fukuda et al. used a fully connected recurrent neural network, optimized by a genetic algorithm, to determine the joint trajectory of a biped robot during both single and double support phases of walking [27]. The sensory input provided to the network was the position of the zero moment point (ZMP) [116] of the robot as calculated from force sensors on the feet, and the outputs were the desired joint angles and velocities of the robot which maintain the ZMP within the support area. This method was shown to be able to produce 3D walking in a 13 DOF biped robot. A similar approach was used by Reil in 1998 where a fully connected recurrent neural network with 10 neurons, was used to control a 6 DOF biped robot in physics based simulation, by optimizing the weights of the network using a genetic algorithm [98]. The Honda P2 and P3, developed by Hirai et al. [45] and presented in 1998, also walked by specifying prerecorded joint angle trajectories, which were measured from motion capture data of human walking. However, there were three feedback controllers which implemented a measure of online adaptivity, in the face of environmental and modelling uncertainties. A model ZMP controller was used to regulate the desired position of the ZMP if the robot was becoming unstable. A ground reaction force controller was used to modify the joint angle trajectories to acheive the desired ZMP. Finally, a foot placement controller was used to correct for possible instability produced by the change in joint angle trajectories. Using this method, the robot was able to walk on flat surfaces and small inclines. There are only a few models which have attempted a more accurate representation of the 6
Figure 1-4: Robots which include passive dynamics. (a) Passive dynamic walker by McGeer (top left) (b) BAPS (top right) (c) MIKE (bottom left) (d) Spring Flamingo (bottom right)
7
neural pathways of the spinal cord, including central pattern generator circuits and reflex pathways known to be involved in locomotion. The model of Wadden and Ekeberg, 1998 [117] is a 2D simulation of one leg, with two degrees of freedom. Their model has a hierarchical architecture in which reflex pathways are at the lower level, modulated by the neural pattern generator (NPG) at the higher level. The neural pattern generator has four states, corresponding to four phases of gait: propulsion, lift-off, swing and touchdown. In the absence of input, the pattern generator transitions between the phases at fixed time intervals, but sensory input can modify the duration of the phases. A single higher order neuron regulates the output of the NPG, as well as the strength of muscle activation to alter the frequency of stepping. The hip position is used to switch between swing and stance phase. Foot contact information is used to transfer between touchdown to propulsion phase. In addition each muscle has a fast feedback pathway which simulates the stretch reflex. In 2001 Ogihara et al also developed a 2D model, but with seven links [76]. The model includes monoarticular and biarticular muscles. In this model, every joint has a central pattern generator, as well as reciprocal innervation based on proprioceptive information. The muscle spindles are connected to the own muscle, they inhibit the antagonist, and facilitate activation in synergistic biarticular muscles. The golgi tendon organs are assumed only to inhibit the corresponding muscle. The CPG has no sensory input. The cutaneous reflex which inhibits the flexors and excites the extensor of the same leg during foot contact. They used a genetic algorithm to tune the weights of the network, and were able to achieve a walking pattern which closely resembled the walking pattern of a human in its joint angle trajectories and ground reaction forces, but deviated to some extent in its pattern of muscle tensions. Rybak also developed a 2D neuro-musculo-skeletal model in 2002 [102]. Each leg had 3 links modelled as the hindlimbs of a quadruped. In this model, each muscle is controlled by a neural module. Each neural module has an alpha motoneuron, a Renshaw cell, Ia and Ib inteneurons and two interneurons which are reciprocally inhibited to form a CPG. The Ib force dependant input is used to inhibit the motoneuron of the same side. The Ia is used to inhibit the activation of the alpha-motoneuron of the antagonist muscle. In addition, all the interneurons are reciprocally inhibitorily connected to their counterparts in the neural module of the antagonist muscle. All, the sensory inputs, including cutaneous inputs also go to the two interneurons of the CPG. Using this structure, the model is shown to be able to produce stable stepping. These biomimetic models go a long way towards illustrating the potential architecture and role of spinal circuits in locomotion. Although completely different in character, another kind of model has provided great insight into the control of locomotion. These are the category of machines called passive dynamic walkers, and their actuated variations. While the previous category of models is characterised by the presence of accurate biomimetic control architectures, passive dynamic walkers derive their explanatory power from the absence of control. One of the first passive dynamic walkers was developed by McGeer in 1990 [69]. This was a lower body biped structure with straight legs and curved feet, which was shown to be able to maintain balance and walk down a slope in the absence of actuation (Fig. 1-4(a)). McGeer also showed that a similar walker including knee joints could perform the same feat. Following this work, it was shown by Goswami et al. [32] and Garcia et al [30] that a biped could also 8
walk without circular feet with a compass like gait. In 2001 Collins et al. [15] showed that the principle could be extended to 3D walking. The models served to demonstrate through real physical examples that a large part of the gait pattern in human walking could possibly result due to the physical dynamics of the system, rather than through explicit control. The principle of passive dynamics was incorporated into actuated biped machines. In 1998, Pratt et al. [88] showed that in the actuated biped robot, Spring Flamingo (Fig. 14(d)), the natural dynamics of the body could be exploited to incorporate a passive swing into the gait pattern. In 1999, van der Linde [115] demonstrated in simulation that a bipedal model with straight legs and circular feet could walk on level ground merely using a reflex driven alternating activation of the hip muscles. Later he developed the robot BAPS (Fig. 1-4(b)) to demonstrate the application of this principle in the real world. Building on this idea, Wisse et al. [120], showed that a lower body biped robot with knees, MIKE (Fig. 14(c)), could also walk using only reflexive foot contact based activation of the hip muscles. Humanoid robots built by Paramonov and Lund [77] showed that upper body actuation could generate passive motion in the lower legs sufficient for walking. The successful incorporation of the natural dynamics of the body into actuated systems, further strengthened the idea that it may be an essential characteristic of human locomotion.
1.3
Motivation
Although progress has been made in understanding the neural basis of locomotion, as described above, much remains to be understood. One area which has not been addressed is the relationship between the dynamics of the body and neural control. Although it has been shown that part of the walking cycle can be achieved as a result of the passive dynamics of the body, it has not been studied how this may play a role in affecting the neural control requirements. This issue, referred to as the morphology and control trade-off by Pfeifer [83][84], is important for understanding the principles behind the neural control of locomotion. There are several issues regarding the neural control of locomotion which are not clear. At the spinal level, the relationships between the sensory pathways are not well understood. As described above, some models assume that the joint angle receptors are responsible for the regulation of stance and swing phase, while others assume that foot contact information is used for this purpose. There are several possibilities for how the system may actually function. It may be that during walking the cutaneous reflexes are active while the joint angle based reflexes are suppressed, or it may be the other way around. A third possibility, is that a higher brain center is responsible for processing the information both from the joint angle and cutaneous sensors to activate the most appropriate reflex response in a given situation. A fourth possibility is that both are always active, and whichever fires first regulates the switch from stance to swing phase. Furthermore, it is not clear which reflex pathways are most important in locomotion. The relative amplitudes of the stretch reflexes, for example, compared to that of the golgi tendon organ based positive force feedback pathways, or to the cutaneous reflex pathways is an issue that is not well understood. Additionally, the relative amplitude of the reflex pathways compared to the CPG is also an open question of interest. 9
The interplay between the CPG and sensory information is also not well known. While it is known that sensory information to the CPG can entrain the system to reach a stable limit cycle, it is not clear which sensors would be most suitable for the task, and how they would be connected to the CPG. Taga assumed that the joint angles were used for CPG input, but they had to be regulated according to the phase of the gait cycle. Since the phase of the gait cycle as defined in the model could only be calculated based on vestibular information, the model assumes that the sensory inputs would be regulated based on supraspinal inputs, which would require long neural latencies and make the system quite slow. The issue of coordination between the CPG and the reflex pathways involved in the regulation of phase is another open question. It is known that primarily the CPG is responsible for regulating the locomotor rhythm, and its frequency can be controlled by supraspinal inputs. However, the cutaneous or joint angle based reflex pathways can also determine locomotor rhythm, as, for example, shown in the robot MIKE [120]. The question is how these two parallel systems of regulating locomotor rhythm are coordinated. All these questions are intrinsically tied to the interaction between the dynamics of the morphology and various neural modules during locomotion. Thus, understanding the relationship between the biped morphology and its control is important from the perspective of gaining insight into the control mechanisms of human locomotion. The goal of this work has been to understand this relationship and apply it to the investigation of human locomotion.
1.4
Research Methods
The collection of papers presented in this thesis use various methods and approaches to address the questions raised above. The issue of the interaction between the body dynamics and neural control, which is a main focus of this work, is investigated using two different approaches. One approach is the use of a commercial physics-based simulation engine which provides fast real-time simulation of robot movement. This environment has been used to simulate biped robots controlled by artificial neural networks, which are optimized using genetic algorithms [61][63]. This approach forms the basis of Chapters 2 and 3. The second approach is the development of real world robots to give insights into possible effects of the physical dynamics of the body on control requirements. This approach forms the basis of Chapters 4 and 5. The investigation of issues related to the neural architecture of the spinal cord, such as the relationship between the reflex pathways, the relative importance of CPG and reflex pathways, and the interaction between the CPG and the reflex pathways have also used two different approaches, although both are based in simulation. The first is the development of a biomimetic neuro-musculo-skeletal model, where the physical characteristics of the body are represented as close to human biomechanics as possible and simulated using accurate rigid-body dynamics, and the neural architecture closely mimics the pathways known from human neurophysiology. This approach forms the basis of Chapter 6. The second approach is the use of physics-based simulation, as described above, using characteristics of the physical body and neural network which are abstracted away from the details of the real system, in order to enable the investigation of a single issue. This approach forms the basis 10
of Chapter 7. The sequence in which these investigations were performed was not completely incremental or linear. The development of the robots, proceeded in parallel with the development of the neuro-musculo-skeletal model, as well as the work done in physics based simulation. This thesis represents the culmination of all these separate threads of work, and is a unique opportunity to present an overview of the entire collection of results. Although the conception of the project was clinical, it was highly interdisciplinary in nature and the contributions of the project were more far-reaching than just clinical science. It had relevance for the fields of biology, robotics, biomechanics, adaptive behavior and artificial intelligence among others. This was considered positive, and it was encouraged to present the multi-faceted contributions of the work to various fields. Thus, the publications were heterogenous in their style and content. Some of the publications were geared towards robotics conferences and focused on engineering aspects of the work, while others were for adaptive behavior or biology audiences and focused on insights into natural locomotion. The function of the last chapter therefore is to summarize all the results and discuss their implications as a whole for understanding human locomotion.
1.5
Outline of the Thesis
The body of the thesis is composed of seven publications, five conference papers and two journal papers, each of which is presented as a separate chapter. For improved readability the papers are organized not according to chronology but content, and are placed under four sub-topics containing two or three papers each. Part I focuses on the topic of morphology and its effects on control, studied in simulation and theory. Part II focuses on the relationship between the effect of the upper body on the control of the lower body, through the development of real-world robots. Part III investigates the relationship between central pattern generators and sensorimotor reflexes, using simulation. This is followed by the last chapter, which will contain a summary of the results and discussion.
11
Overview 9 Chapter 1: Introduction
Part I: Morphology
9 Chapter 2: Effect of Morphological Change Paul, C. and J. C. Bongard (2001) ”The Road Less Travelled: Morphology in the Optimization of Biped Robot Locomotion”, in Proceedings of The IEEE/RSJ International Conference on Intelligent Robots and Systems, Maui, Hawaii, USA.
9 Chapter 3: Neural Coupling through Morphology Paul, C. (2003) ”Bilateral Decoupling in the Neural Control of Biped Locomotion”, in Proc. 2nd International Symposium on Adaptive Motion of Animals and Machines, Kyoto, Japan.
9 Chapter 4: Computational Role of Morphology Paul, C. (2004) ”Morphology and Computation” 8th Internation Conference on Simulation of Adaptive Behavior, Los Angeles, CA, USA.
Part II: Upper Body
9 Chapter 5: The STUMPHopping Robot Paul, C., Iida, F., and R. Dravid1 (2002) ”Design and Control of a Pendulum Driven Hopping Robot ”, in IEEE/RSJ International Conference on Intelligent Robots and Systems Lausanne, Switzerland
9 Chapter 6: The BENDY Walking Robot Paul, C., Yokoi H., and K. Matsushita (2004) “Design and Control of Humanoid Robot Locomotion with Passive Legs and Upper Body Actuation”, International Symposium on Robotics Paris, France.
1
All three authors have equal contributions to this paper
12
Part III: Spinal Control
9 Chapter 7: Neuro-Musculo-Skeletal Model Paul, C., Bellotti, M., Jerzernik, S. and A. Curt (prep) “Development of a Human Neuro-Musculo-Skeletal Model for Investigation of Spinal Cord Injury”, to Biological Cybernetics
9 Chapter 8: Sensorimotor Control Paul, C. (accepted) “Sensorimotor Control of Biped Locomotion”, Journal of Adaptive Behavior
9 Chapter 9: Discussion
13
Part I Morphology
14
Chapter 2 Effect of Morphological Change Morphology in the Optimization of Biped Robot Locomotion1 In this paper, stable bipedal locomotion has been achieved using coupled evolution of morphology and control on a 5-link biped robot in a physics-based simulation environment. The robot was controlled by a closed loop recurrent neural network controller. The goal was to study the effect of macroscopic, midrange and microscopic changes in mass distribution along the biped skeleton to ascertain whether optimal morphology and control pairs could be discovered. The sensor-motor coupling determined that small changes in morphology manifest themselves as large changes in the performance of the biped, which were exploited by the optimization process. In this way, mechanical design and controller optimization were reduced to a single process, and more mutually optimized designs resulted. This work points to alternative routes for efficient automated and manual biped optimization.
2.1
Introduction
In the traditional approach to biped design and optimization, designers usually preselect a morphology. They then design a controller for the preselected mechanical design. Often, the biped and the controller are implemented in simulation to predetermine if together they lead to the desired performance, before the actual robot is built. Finally, the robot is built in the real world, and the controller parameters are tuned. This methodology has led to many successful biped walking robots such as the Spring Flamingo[88], the Honda Humanoid[45] and the BIP2000[26]. The success of this methodology has encouraged robot designers to design morphologies first and then pursue further performance improvements with controller enhancements. This methodology has also been adopted in evolutionary robotics. There are numerous examples of biped controller design and optimization using genetic algorithms2, where the biped morphology has been preselected and fixed. For 1
Paul, C. and J. C. Bongard (2001) ”The Road Less Travelled: Morphology in the Optimization of Biped Robot Locomotion”, in Proceedings of The IEEE/RSJ International Conference on Intelligent Robots and Systems, Maui, Hawaii, USA. 2 abbreviated as GA in following text
15
example, Fukuda et al.[27] use a GA for stabilization control of a simulated fixed morphology biped robot. Similarly, Reil[98] and Hase et al.[37] use genetic algorithms to evolve the weights of neural controllers for walking in fixed simulated bipeds. The strong focus on control has left a second avenue, that of morphology enhancements leading to large performance increases in active walking, relatively unexplored. In recent times, advances in Embodied AI have increasingly stressed the importance of the robot body [9] and morphological parameters, such as mechanical structure and sensor and actuator placements, which greatly influence the performance of a robot [83]. More significantly, work in passive dynamic walking by McGeer[69], Collins et al.[15] and Goswami et al.[32] have shown that morphology is all important in determining the performance of passive biped walking mechanisms. However, this effect of morphology has not been systematically studied or harnessed to tune performance in the domain of active biped robot locomotion. Explorative studies of the coevolution of morphology and control have addressed the issue indirectly in the past. Pioneering work on the evolution of morphologies and neural controllers was conducted in a virtual physics based simulation environment by Sims[108] and made clear the strong interdependence between morphology and control. Similar work in coevolving controllers and body plans for obstacle avoidance in simulation followed by Lee et al.[61] and for real world robots by Lipson and Pollack[65]. The effect of morphological attributes specifically on locomotive performance was studied by Bongard and Paul[3] and Chocron and Bidaud[13]. While these projects provide the theoretical inspiration for this work, their loosely constrained morphologies prevent any direct association with issues in biped locomotion. More recently, this methodology of jointly optimizing morphology and control was applied to develop a pseudo biped robot, in which the two moving legs driven by open loop oscillators were stabilized by a long flat tail [51]. However, the effects of morphology on control have not yet been truly integrated into the primary design process of more anthropomorphic biped robots. In this paper, for the first time morphological parameters and a closed loop controller have been optimized simultaneously for a biped robot, so that the mechanical and controller design are carried out in a single process. To demonstrate the use of this methodology, we address the engineering design problem of mass distribution along the biped skeleton, one of the first mechanical design decisions. The distribution of the mass is represented as a problem of parametrically positioning discrete blocks along the length of the biped, and determining their geometrical dimensions. This problem was chosen because it was analogous to designer decisions regarding relative positions of motors and gears, usually the heaviest components, along the biped skeleton. Thus the goal was to combine the basic biped design and controller optimization process, so that the controller could take advantage of the effect of changing mass distributions on the dynamics of the biped. We show that this is a successful method for finding stable biped locomotion.
16
Table 2.1: Morphological parameters of the agent. A ul represents one unit length defined as the radius of the spherical sockets at the hip and knees. A um represents one unit mass defined as the mass of the same spherical socket. The parameters in boldface vary under evolutionary control between the ranges shown below. Index Object Dimensions Mass 1 2 3 4 5 6 7
Knees Hip sockets Feet Lower Legs Upper Legs Waist Waist Mass Block
1 um 1 um 1 um 0.25 um 0.25 um 0.25 um L = [0.001, 0.246]um M = [0.002, 0.505]um H = [0.004, 1.009]um 8 Lower Mass Block l = [0.4, 3.6] ul L = [0.001, 0.246]um w = h = [0.2,3.0] ul M = [0.002, 0.505]um H = [0.004, 1.009]um 9 Upper Mass Block l = w = [0.2,3.0] ul L = [0.001, 0.246]um h = [0.4, 3.6] ul M = [0.002, 0.505]um H = [0.004, 1.009]um Index Joint 10 Knee 11 Hip 12 Hip
r = 1 ul r = 1 ul r = 2 ul, w = 3 ul r = 0.5 ul, h = 8 ul r = 0.5 ul, h = 8 ul r = 0.5 ul, w = 8 ul l = [0.4, 3.6] ul w = h = [0.2,3.0] ul
Plane of Rotation
Range of Motion
sagittal sagittal frontal
:<; >= (radians) :<; ? = ; ? :A@; = A@;
17
Figure 2-1: Biped construction: Fig. a) shows the biped skeleton and its 6 degrees of freedom Fig. b) shows the biped with attached mass blocks. (T and P refer to tactile and proprioceptive sensors respectively, and M indicates a motor.)
2.2
The Robot
The robot is a 5-link biped robot with 6 degrees of freedom, simulated in a real-time, physics-based virtual environment called MathEngine 3 . The robot has a waist, two upper leg and two lower leg links as shown in Fig 2-14 . Each knee joint, connecting the upper and lower leg links, has one degree of freedom in the sagittal plane. Each hip joint, connecting the upper leg to the waist, has two degrees of freedom: one in the sagittal plane and one in the frontal plane. These correspond to the roll and pitch motions. The joints are limited in their motion with joint stops, with ranges of motion closely resembling those of human walking. The hip roll joint on each side has a range of motion between :CBEDGF and BEDF degrees with respect to the frontal plane. The hip pitch joint has a range of motion between :BED and BED % , with respect to the sagittal plane. The knee joint has a range of motion between :CBEDGH and with respect to the axis of the upper leg link to which it is attached. Each of the joints is moved by a simulated torsional actuator. The actuator receives position commands from the controller. It uses proportional control to determine the velocity of the link, with a relatively low maximum torque ceiling. The torque applied to actuators 3
developed by MathEngine PLC, Oxford, UK, www.mathengine.com The software incorporates physical forces corresponding to gravity, friction, contacts and collisions in simulating rigid-body dynamics. 4 The lower leg links are mounted on cylindrical feet, with a relatively large radius of curvature, to approximate the function of the ankle and feet in humans. This is similar to the design of passive dynamic walkers.
18
INPUT LAYER
T1
T2
P1
P2
P3
P4
P5
P6
B1
B2 HIDDEN LAYER
M1
M2
M3
M4
M5
M6
OUTPUT LAYER
Figure 2-2: Pictorial representation of the neural network used to control both types of agents. T1 and T2 correspond to the two touch sensors, P1 through P6 indicate the six proprioceptive sensors, and M1 through M6 indicate the six torsional motors of the biped. B1 and B2 indicate the two bias neurons included in the network. is determined by
Y[Z\Z I[]R^`_ Z MLON*PRQTSUQWV J :X, Q : (2.1) Y where is the actual joint angle, is the desired joint angle, IK]R^a_ is the maximum torque ceiling, VX , and S is the feedback gain matrix. IKJ
This means that the velocity of a link will be greater the further it is from the commanded joint angle position, but if the force required to achieve this velocity is too large, it will only apply the maximum force. This mechanism incorporates a measure of compliance into the system, and is in accordance with the capabilities of real world actuators.
2.3
Neural Controller
The neural controller used is a recurrent neural network, which due to its lateral inhibitory and excitatory connections in the hidden layer, provides the intrinsic capability of producing cyclic dynamics. While this network cannot be as flexibly frequency entrained as the neural oscillator of Taga et al.[111], which is defined by a system of differential equations, it is much less computationally intensive. The choice of this network is based on the work of Gallagher et al.[29] and Reil[98], in which it has been shown capable of producing openloop cyclic output for biped walking. The agent contains two haptic sensors in the feet, and six proprioceptive sensors and torsional actuators attached to the six joints, as outlined in Figs. 2-1 a) and b). At each time step of the simulation, agent action is generated by the 19
propagation of sensory input through the network shown in Fig. 2-2 and the values of the output layer are fed into the actuators as desired positions. The input layer contains nine neurons, with eight corresponding to the sensors, and an additional bias neuron. All neurons in the network emit a signal between : and : the haptic sensors output if the foot is in contact with the ground, and : otherwise; the proprioceptive sensor values are scaled to the range bc: G ed depending on their corresponding joint’s range of motion; and bias neurons emit a constant signal of . The input layer is fully connected to a hidden layer composed of three neurons5. The hidden layer is fully and recurrently connected, plus an additional bias neuron. The hidden and bias neurons are fully connected to the eight neurons in the output layer. The activations of the hidden and output neurons are computed by
opi where
Nf
opi
hi
inmopi g
j lk
(2.2)
is the output of a neuron in the previous layer. In the hidden layer, however, represents both the output from the input layerinm at the current time step, and the output from the hidden layer at the previous time step. is the weight of the synapse connecting k them. The output of this neuron is then given by
o
H ^ : rq!s
(2.3)
The values at the output layer are scaled to fit the range of their corresponding joint’s range of motion. Torsion is then applied at each joint to attain the desired joint angle.
2.4
The Genetic Algorithm
A fixed length genetic algorithm was used to evolve the controllers reported in this paper. Each run of the genetic algorithm was conducted for 300 generations, using a population size of 300. At the end of each generation, the ) most fit genomes were preserved; the others were deleted. Tournament selection with a tournament size of three, is employed to probabilistically select genotypes from among those remaining for mutation and crossover. t pairwise one-point crossings produce FGu new genotypes: the remaining Fv( new genotypes are mutated copies of genotypes from the previous generation. The mutation rate was set to generate an average of five mutations for each new genome created.6 Mutation involved the replacement of a single value with a new random value.7 Each genome contains floatingpoint values encoding the u synaptic weights of the neural network, plus any additional 5
A hidden layer of less than three neurons was experimentally found to be insufficient for the task. The mutation rate was determined experimentally to yield high performance. 7 This enabled the algorithm to take larger steps in the fitness landscape as compared to adding a proportion of the current value. 6
20
morphological parameters8 . These values are rounded to two decimal places and range between : 1w and 1w . Each individual is evaluated for 2000 time steps of the dynamics simulation. The initial condition for each individual at the first time step is one in which all joint angles and velocities are set to zero. This results in a fully upright posture, with all parts aligned in the frontal plane, and both feet at an equal distance from the target. The evaluation is prematurely terminated if the center of gravity of the waist drops below the original vertical position of its knees (it falls) or if both feet lift off the ground (it starts to run). This second termination criteria was added because the primary interest of this project was to study walking, and not running gaits. At the end of the evaluation, the distance of the biped travelled in the sagittal plane (determined relative to its original position) was considered its fitness.
2.5
Experiments
In order to study the effect of a varying range of mass distributions on the performance of the biped three sets of experiments were conducted. In each of the three experiments, the total mass of the blocks distributed along the skeleton represented a different fraction of the total mass of the biped. We wanted to study the effect of macroscopic, midrange and microscopic changes in weight distribution of the biped skeleton to study whether optimal morphology/control pairs could be discovered over a range of mass distributions. The lengths, widths and vertical attachment points of the blocks of the lower and upper leg, and the length and width of the block on the waist were supplied as the morphological parameters subject to evolutionary control. The block on the waist was always fixed at the midpoint of the waist to maintain bilateral symmetry. The rectangular blocks on the legs could be as long as the leg link to which they were attached, and were not allowed to extend beyond the endpoint of the link. There were always two blocks per leg, each of which could be attached to the upper or lower leg link. The size and mass ranges for the blocks are summarized in Table 2.1. Although the sizes of the blocks varied, the total mass of the blocks added together was kept constant. Thus the goal was to study the effect of mass distribution of the same total mass along the body. In the first set of experiments with microscopic changes in mass distribution, the blocks were allowed the same range of geometric variability as described above but their total mass was normalized to 0.25 um. Thus, the mass of the blocks represented 6% of the total mass of the biped. In this second set of experiments with mid-range changes, the blocks were now normalized to a total weight of 0.513 um. This represented 13% of the total weight of the biped. In the final set with macroscopic changes, the total mass of the blocks was 1.025um, representing 27.3% of the total mass of the biped. In each of the three cases, 20 evolutionary runs were conducted, with one run lasting approximately 40 minutes. Thus the results from 40 hours of data collection are presented below. 8
The morphological parameters were spliced in at regular intervals between the synaptic weights in order to ensure an even distribution for crossover.
21
2.6 2.6.1
Results Microscopic Changes
In the case of microscopic changes in weight distribution, 20 runs were performed. Of these runs, only one evolved stable locomotion (See Fig 2-3(a)). This is not a surprising result, as the problem of achieving a stable limit cycle in bipedal locomotion is not trivial. The best biped in this set of experiments had a morphology optimized to the configuration as shown in Fig. 2-3(b), which was largely different from the initial setting of its morphology. Both leg blocks were fixed to the upper leg link. One block was thicker and shorter and was positioned higher on the leg, and the other narrower but longer was positioned a bit lower on the same link. The waist block is also considerably large, leading to a vertical center of mass at 0.542 ul. The gait of this biped demonstrated a rapid shuffling motion.
Figure 2-3: (a) Best fitness achieved in each generation with microscopic mass distribution changes (b) Morphology of most fit biped achieved with microscopic mass distribution changes
2.6.2
Mid-Range Changes
In the case of mid-range mass distribution changes, 20 runs were also performed. Of these runs, 2 evolved stable locomotion (See Fig. 2-4(a)). The maximum fitness attained by the best biped was 44.2 ul. This biped had a morphology as shown in Figure 2-4(b), which was also largely different from its initial setting. In this case again, both the leg blocks are positioned on the upper leg. One block is thick and long, hiding the other block inside it which is thinner and a bit shorter. The waist mass is also considerably large.
2.6.3
Macroscopic Changes
Finally, in the case of macroscopic changes to the mass distribution, 20 runs were also performed. Of these three evolved stable locomotion. The best fitness achieved in this case 22
Figure 2-4: (a) Best fitness achieved in each generation with mid-range mass distribution changes (b) Morphology of most fit biped achieved with mid-range mass distribution changes was 32.1 ul as shown in Figure 2-5(a). This run produced the agent shown in Figure 2-5(b). In this case, the waist mass has been reduced. Conversely, the upper legs on which the two blocks are again positioned (one inside the other), are even heavier.
2.7
Discussion
Stable Locomotion Using Morphology The experimental results have shown for the first time that stable locomotion can be implemented using coupled optimization of morphology and control in biped robots. Six successful examples of this process have been charted here. This shows that the success of our methodology is not infrequent or difficult to achieve. The results also show that this methodology is robust in the face of varying ranges of morphological parameter settings. Morphology for Performance Optimization In each of the three cases shown, the final morphology of the biped is largely different from the configuration of the best biped at the beginning of the run. However, although the range of potential variability in the size and placement of the blocks and the vertical position of the center of mass was large (between 0.4 and 0.6), the morphologies of the three best walkers all had no blocks on their lower legs, and their vertical CoM positions fell in a narrow band between 0.54 and 0.56 (Fig. 2-6). This suggested that within the space of morphological variability a certain region was more likely to yield stable locomotion than others and that the morphology was not entirely inconsequential. Comparing the three graphs of best fitnesses in Figures 2-3(a), 2-4(a) and 2-5(a), another significant difference is observed. In the microscopic case, there is only one case of stable locomotion. In the mid-range case, two cases of stable locomotion are seen and in the macroscopic case, three cases were discovered. Unless these results are purely coincidental, it suggests that the greater the possibility for morphological change, the greater is 23
Figure 2-5: (a) Best fitness achieved in each generation with macroscopic mass distribution changes (b) Morphology of most fit biped achieved with macroscopic mass distribution changes the likelihood of achieving stable locomotion. While these results indirectly suggest the importance of morphology in performance optimization, more concrete evidence was collected: in some of the runs, the trajectories of the centers of mass of the bipeds were tracked during their evaluation. In several of these, it was seen that morphological mutations significantly improve performance. In one population, a mid-range biped was tracked during its life history: its center of mass reached a distance of 12 ul. It was replaced by its child, which sustained eight point mutations, and its center of mass travelled H 1w ul in the desired direction. The trajectories of the centers of mass of these two agents are indicated in Fig. 2-7 by the light gray and dark gray lines, respectively. Of the eight mutations, one of these was a morphological change. A third agent was tested, which was genotypically equivalent to the more fit child, except that the morphological mutation was suppressed. This third agent’s center of mass reached a distance of ) 1 u ul, and its trajectory is indicated by the black line in Fig. 2-7. The control parameter mutations serve to stabilize the gait and to straighten the biped trajectory to some extent. The morphological mutation further corrects the robot’s direction of travel, and increases its total distance travelled. It is thus clear from the results that morphological change can be used by an optimization algorithm to tune robot performance. It is no surprise that this should be the case. In a closed loop system such as the one implemented here, the controller recieves sensory input from the biped mechanical system, such as joint angle and foot contact information. This sensory input is processed through the network, and position commands are sent to the motors. The position control is not precise and rigid, but includes a large measure of compliance so that small mechanical changes affect the actual joint angle positions, with more severe changes affecting foot contact. This in turn affects the next set of sensory inputs to the controller and the next set of motor commands to the biped. Thus the controller and the morphology form an inseparable closed loop dynamical system, in which a slight change to any parameter can have large repercussions on the overall behavior of the system. What is surprising, however, is
24
Figure 2-6: Changes in vertical position of biped Center of Mass: The thin line tracks the changes of the vertical CoM for the most fit biped in the microscopic case. The middle line tracks the CoM of the best biped in the mid-range case and the thick line is that of the best biped in the macroscopic case. that this effect has not been utilized more effectively in biped robot optimization. Performance vs. Efficiency The results here show concretely that morphological parameters can be used to optimize performance in a biped robot. What they do not necessarily suggest is that the performance improvement gained from a morphological change will be larger than that of a controller change. The consideration of a morphological change is therefore not based on performance, but on efficiency. Sometimes, making a simple change in weight distribution on a biped robot (by attaching weights, for example) can be much more efficient than reprogramming a controller, especially when the very same controller would suffice with a slightly altered morphology. Thus, including morphology in the optimization process can lead to much more efficient design in some cases. Combined Mechanical and Controller Design In addition to the fact that including morphology in the optimization process leads to greater efficiency, if included from the beginning of a design project it can also help to conflate the two phases of robot design into one process. When mechanical design considerations, such as the placement of motors and gears in this case, are included in the combined optimization process, the end products are not only an optimized controller, but also a prototype mechanical design which is highly tuned to the controller. Implications for Future Biped Design It is hoped that this research will lead the way to further investigations into the use of morphology for optimizing robot performance. While this research uses a genetic algorithm for 25
Figure 2-7: Trajectories of CoM of three Bipeds: The light gray trajectory shows the performance of an evolved biped. The dark gray line shows the performance of one of its children, which has seven control and one morphological mutation. The black trajectory shows the performance of a third biped, which is equivalent to the more fit child, except that the morphological mutation is suppressed. automated robot design, the implications of this work are not limited to GAs or automated robot design. In fact, the use of other techniques may allow more quantitative analysis of why a particular morphology and control leads to stable walking. The broader implications of this research point to possibilities for optimization of biped robot performance through morphology. Currently, when biped robot performance does not meet designer expectations, engineers and programmers spend all their effort implementing controller changes until the robot’s performance is as desired. It is at this juncture that we would like to encourage engineers to consider the alternative route of making morphology modifications9. As has been shown in this work, often a simple morphology modification can lead to a large increase in performance, equivalent to several man hours of reprogramming. While it may be argued that such morphological changes to a robot may be much more work intensive than software alterations, this is simply because biped robots in the past have been built with preconceived morphological finality and have been very difficult to alter post hoc. But this need not be the case. If future biped designs allow for greater morphological flexibility, more alternative routes to successful bipeds could be discovered during the optimization process. The insights gained from this methodology will be used for the design of a real physical biped robot at the Artificial Intelligence Lab, University of Zurich. The biped will have 8 degrees of freedom, and the simulation prototype will be extended to include additional parameters to further constrain the robot design. This will ensure that the biped is optimized to perform its basic locomotion with a simple neural network controller architecture 9
In addition to parameters such as mass distribution, link length and placement of sensors and actuators, this could also include passive material properties such as changes in joint stiffness, contact surfaces, or placement of soft tissues for damping.
26
similar to that shown here. Additional morphological parameters will be included in the optimization of robustness of the robot against both non-level walking surfaces and external disturbances.
2.8
Conclusions
This paper outlines a new approach to the optimization of biped robot locomotion: inclusion of morphological parameters in the optimization process. We describe three sets of experiments in which morphological parameters pertaining to mass distribution are considered coupled with control parameters in the evolution of stable bipedal gaits. The methodology shows how mechanical design decisions and controller optimization can be accomplished in a single step and can lead to more mutually optimized systems. This work opens up the possibility for robot designers to make iterative or systematic post hoc morphological changes to further tune biped performance.
27
Chapter 3 Neural Coupling through Morphology Bilateral Decoupling in the Neural Control of Biped Locomotion1 In this paper, a bilaterally decoupled neural controller was used to achieve stable locomotion in a 6 DOF biped robot with 5 links, implemented in a virtual physics-based simulation environment. The neural controller consists of an independent neural rhythm generator for each leg. The rhythm generator is a recurrent neural network, whose weights are optimized by a genetic algorithm and are identical for both legs. Each one receives inputs only from its own joint angle sensors and touch sensor on the foot, and produces motor commands for joints of the same leg. However, by using sensory feedback from the mechanical coupling of the legs through the body, they can synchronize to produce coordinated stepping movements. Furthermore, improvements in the quality of the gait can be achieved only with global sensory information. The results suggest the possibility that there may not be a need for explicit contralateral sensory coupling or neural connections in bipedal locomotion.
3.1
Introduction
Neural control of human locomotion is not yet fully understood. There is strong evidence which suggests that isolated neural circuits called central pattern generators exist in vertebrates to produce rhythmic oscillatory patterns, which coupled with the dynamics of the body produce locomotion. Experiments with cats have shown that a decerebrated cat can walk on a treadmill2 when some parts of the brain stem are stimulated [107]. It has also been shown that even in the complete absence of stimulus neural circuits exist in the spinal cord which can produce oscillatory motor outputs [33]. Similar results have been found in humans. In experiments with newborn infants it has been shown that they are capable of producing stepping movements when they are held upright [114]. Also, in experiments with complete adult paraplegic patients, it is shown 1
Paul, C. (2003) ”Bilateral Decoupling in the Neural Control of Biped Locomotion”, in Proc. 2nd International Symposium on Adaptive Motion of Animals and Machines, Kyoto, Japan. 2 if the body is suspended
28
that they are able to produce stepping movements when supported on a moving treadmill [20]. These results suggest that the same neural circuits which are capable of autonomous rhythm generation in vertebrates may be involved in human bipedal locomotion. Although computational studies have shown that small neural circuits consisting of 2-6 mutually inhibiting neurons are capable of such autonomous rhythm generation [68], the half-center hypothesis of central pattern generation [10] is most prevalent for vertebrates. This proposes that small neural units of two reciprocally inhibited “half-center” neurons are responsible for rhythm generation in each joint. The half-center model of a CPG has been implemented by Matsuoka as a set of differential equations [67], which has been widely used in studies of rhythmic movement generation. Kimura et al [57] developed a neural controller for a quadruped robot, consisting of one such CPG for each of the hip and knee joints. The hip CPGs were connected to each other so that diagonal legs were paired. Taga used the Matsuoka oscillator in a similar neural controller for biped locomotion, with a single CPG connected to the flexor and extensor muscles of each of the hip, knee and ankle joints. [111]. In this model, explicit phase dependent connections were introduced between the ipsilateral oscillators, and permanent inhibitory connections were implemented between contralateral oscillators of each joint to ensure phase-locking in an anti-phase relationship. However, in a model of arm control using CPGs it was shown that phase-locking could occur only by sensing of mechanical coupling[118].
Table 3.1: Morphological parameters of the biped robot. A ul represents one unit length defined as the five times the radius of the spherical sockets at the hip and knees. A um represents one unit mass defined as the mass of the same spherical socket. The ranges of the parameters which vary under evolutionary control are shown in square brackets. Index Object Dimensions Mass 1 2 3 4 5 6
Knees Hip sockets Feet Lower Legs Upper Legs Waist
Index Joint 10 Knee 11 Hip 12 Hip
r = 0.2 ul r = 0.2 ul r = 0.4 ul, w = 0.8 ul r = [0.04, 0.16] ul, h = 8 ul r = [0.04, 0.16] ul, h = 8 ul r = [0.04, 0.16] ul, w = 8 ul
1 um 1 um 1 um [0.1, 0.4] um [0.1, 0.4] um [0.1, 0.4] um
Plane of Rotation
Range of Motion
sagittal sagittal frontal
: ; x= (radians) :y; ? = ; ? :zA@; = ;
In this paper, the goal was to investigate whether explicit contralateral connections, as used in the previous studies on biped locomotion, are necessary for phase-locking. It was hypothesized that the mechanical coupling of the legs through the body could be sufficient to coordinate the movements of the two legs. To test this, an independent neural rhythm 29
generator was used for each leg of a 6 DOF biped robot. The rhythm generator was a recurrent neural network with one hidden layer, which only received sensory inputs from one side of the body. The weights of the network were optimized using a genetic algorithm, as in the work of Ogihara et al. [76], but the search space was additionally improved using morphological parameters [4]. Two different neural architectures were tested: one with complete bilateral decoupling, where only sensory information related to the legs was available to each rhythm generator, and another in which global sensory information related to body attitude was also provided. It was shown that in both cases, the legs were able to coordinate despite the absence of explicit neural connections and produce stable walking. The following sections, describe the biped robot (Section 2), the neural controllers with bilateral decoupling and global sensing (Section 3) and the genetic algorithm (Section 4). Section 6 presents the main results of the experiments, followed by Section 7 which discusses the results. Section 8 summarizes with conclusions.
3.2
The Robot
The robot is a 5-link biped robot with 6 degrees of freedom, simulated in a real-time, physics-based virtual environment3. The robot has a waist, two upper leg and two lower leg links as shown in Fig 3-1. Each knee joint, connecting the upper and lower leg links, has one degree of freedom in the sagittal plane. Each hip joint, connecting the upper leg to the waist, has two degrees of freedom: one in the sagittal plane and one in the frontal plane. These correspond to the pitch and roll motions. The joints are limited in their motion with joint stops, with ranges of motion closely resembling those of human walking. The hip pitch joint on each side has a range of motion between :CB{DF and B{DF degrees with respect to the frontal plane. The hip roll joint has a range of motion between :BED H and BED H , with respect to the sagittal plane. The knee joint has a range of motion between :CBEDGH and with respect to the axis of the upper leg link to which it is attached. Each of the joints is moved by a simulated torsional actuator. The actuator receives position commands from the controller. It uses proportional control to determine the velocity of the link, with a relatively low maximum torque ceiling. The torque applied to actuators is determined by
Y[Z\Z I[]R^`_ Z MLON*PRQTSUQWV J :X, Q : (3.1) Y where is the actual joint angle, is the desired joint angle, IK]R^a_ is the maximum torque ceiling, VX , and S is the feedback gain matrix. IKJ
This means that the velocity of a link will be greater the further it is from the commanded joint angle position, but if the force required to achieve this velocity is too large, it will only apply the maximum force. This mechanism incorporates a measure of compliance into the system, and is in accordance with the capabilities of real world actuators. 3
MathEngine PLC, Oxford, UK, www.mathengine.com
30
Figure 3-1: The biped physical structure and its six degrees of freedom. The labels indicate the joint angle and foot contact sensors. The widths of the biped links vary under evolutionary control within the range shown in Table 1.
3.3 3.3.1
Neural Controller Complete Bilateral Decoupling
For the investigation of complete bilateral decoupling, a neural controller was designed with an independent identical neural rhythm generators for each leg (Fig. 3-2). Each rhythm generator was a recurrent neural network consisting of an input layer with sensory input nodes and a bias neuron, a hidden layer of 4 nodes with lateral connections and a second bias neuron, and an output layer with motor outputs. The sensory inputs of the network were from the biped robot. The biped has proprioceptive sensors at each joint whose values are scaled to the range b|: [d . It also has haptic sensors on the feet, which outputs if the foot is in contact with the ground and : otherwise. It is actuated by torsional actuators attached to the six joints. Thus, the neural rhythm generators have 4 input nodes, one for each of the three proprioceptive sensors and one for the haptic sensor of a leg. They have 3 output nodes, one for each of the actuated joints of the same leg. The input layer is fully connected to a hidden layer. The hidden layer is fully and recurrently connected, plus an additional bias neuron. The hidden and bias neurons are fully connected to the three neurons in the output layer. All neurons in the network emit a signal between : and . Bias neurons constantly emit a signal of 4 . The activations of the hidden and output neurons are computed by
opi where
opi
Nf
hi
inmopi g
j lk
(3.2)
is the output of a neuron in the previous layer. In the hidden layer, however, represents both the output from the input layer at the current time step, and the output
4
These neurons can be used to regulate the threshold values of neurons in the next layer.
31
Figure 3-2: Bilaterally decoupled neural controller including only joint angle and foot contact sensors. The input nodes for the left leg controller are LR: Left Hip Roll, LP: Left Hip Pitch, LK: Left Knee Pitch, LT: Left foot contact and the outputs are to the left hip roll, hip pitch and knee joints. For the right leg controller the inputs are RR: Right Hip Roll, RP: Right Hip Pitch, RK: Right Knee Pitch, RT: Right Foot Contact and the outputs are to the right hip roll, hip pitch and knee joints. B and B are bias nodes.
inm
from the hidden layer at the previous time step. k them. The output of this neuron is then given by
o
H ^ : rq!s
is the weight of the synapse connecting
(3.3)
The values at the output layer are scaled to fit the range of their corresponding joint’s range of motion. Torsion is then applied at each joint to attain the desired joint angle. The recurrent structure of the hidden layer in the rhythm generator allows for lateral inhibition and thus the intrinsic capability of producing cyclic dynamics. The rhythm generator closely mimics the structure of the monolithic neural controller used in previous work by Paul et al[78] for the evolution of biped walking. This controller received inputs from all joint angle and foot contact sensors in the same network, passed it on to a recurrent hidden layer with 3 nodes, which then passed it on to the output layer with six outputs, one for each joint of the biped. The current network can be visualized as the monolithic network “cut in half” (with a few more nodes added to the hidden layer). The single leg neural rhythm generator can also be considered a simplified version of the ipsilateral leg controller used by Taga [111]. It allows for cyclic dynamics, but with reduced flexibility in the phase relationships between the joints, as there are fewer oscillatory units and the connections in the controller are permanent and not phase dependent.
32
Figure 3-3: Bilaterally decoupled neural controller enhanced with global sensing. In addition to the nodes in the bilaterally decoupled controller in Figure 3-2, the left leg controller has sensory input nodes for WO: waist orientation in transverse plane, WP: waist sagittal position and FP: left foot sagittal position. For the right leg controller the input to FP is the foot position of the right foot instead. However, as the focus of this study was not ipsilateral but contralateral coupling5, the complexity of the network was sufficient.
3.3.2
Bilateral Decoupling with Global Sensing
As the mechanical coupling of the legs determines that movement of either leg will influence global variables corresponding to body attitude, it was hypothesized that including such global sensory information in the network could improve the stability of the gait pattern. Thus, for this test the network described in the previous section was augmented with three new sensory inputs (Fig. 3-3). A global sensor of the orientation of the waist in the tranverse plane (WO) was added to detect deviations of the biped from a straight line path. As the evaluation of a biped was terminated if the biped veered more than 0.9 radians from its initial orientation, the orientation sensor produced a signal in the range [0.0, 0.9]. Two more sensors were included for the foot sagittal position (FP) and waist sagittal position (WP), so if necessary the network could compute the difference between the two, and use it to detect the phase of the gait cycle. These sensors produced output values proportional to position without further normalization. Both the right and left networks were given the three sensor values as additional inputs, so the input layer consisted of seven input nodes instead of four. As the networks were identical, the global sensor values were multiplied 5
coupling between the left and right sides
33
by the same connections weights on both sides, and had the same effect on right and left rhythm generators.
3.3.3
Initial Condition
At the beginning of the simulation all the joint angles and velocities of the biped are 0, and both haptic sensors are 1. Thus, the sensory inputs of the right and left rhythm generators are identical. As the weights of the two rhythm generators are also identical, identical outputs are produced. This could only lead to two situations: standing in place or hopping, which are the two behaviors in which both legs perform identical movements. It would not lead to walking, as the start of walking is an asymmetric motion: one leg enters swing phase as the other one stays on the ground. Thus, the control of the start of walking had to be performed externally to the network. In other words, the initial conditions had to be determined so that the network would function. As the initial conditions are closely coupled to the performance of the network, which is determined by the weights set by the genetic algorithm, it was also necessary to let the genetic algorithm control the initial conditions. Thus, the initial motor commands for the actuators of the biped in the first 10 steps of the simulation are six values generated by the genetic algorithm. The network takes control of the biped after this time has elapsed.
3.4
The Genetic Algorithm
A fixed length genetic algorithm was used to evolve the controllers6 . Each run of the genetic algorithm was conducted for 300 generations, using a population size of 200. At the end of each generation, the % most fit genomes were preserved; the others were deleted. Tournament selection with a tournament size of three, is employed to probabilistically select genotypes from among those remaining for mutation and crossover. H) pairwise one-point crossings produce ) new genotypes: the remaining ) new genotypes are mutated copies of genotypes from the previous generation. The mutation rate was set to generate an average of seven mutations for each new genome created. Mutation involved the replacement of a single value with a new random value. Each genome contains floatingpoint values which are rounded to two decimal places and range between : 1w and 1}G . In the optimization of the fully bilaterally decoupled neural controller the genome encodes ) synaptic weights of the neural network, t morphological parameters to determine the link widths, and u initial position commands, and thus has a total length of u parameters. In the controller with global sensing the number of synaptic weights increases to ut , and the genome length to FGH , due to the three additional input nodes. During evolution each individual is evaluated for 2000 time steps of the dynamics simulation. The initial condition for each individual at the first time step is one in which all joint angles and velocities are set to zero. This results in a fully upright posture, with all parts aligned in the frontal plane, and both feet at an equal distance from the target. The 6
The genome string consisted of floating point values representing the synaptic weights of the network, as well as morphological parameters corresponding to the upper leg, lower leg and waist link widths.
34
No Coupling 10
9
8
Best Fitness
7
6
5
4
3
2
1
0
50
100
150 Generations
200
250
300
Figure 3-4: Best fitness in each generation, for the six evolutionary optimization experiments with complete bilateral decoupling. evaluation is prematurely terminated if the center of gravity of the waist drops below the original vertical position of its knees (it falls) or if it “twists” too much, or if both feet lift off the ground, (it starts to run). This third termination criteria was added because the primary interest of this project was to study walking, and not running gaits. At the end of the evaluation, the distance of the biped traveled in the sagittal plane (determined relative to its original position) is considered its fitness.
3.5
Results
Six full evolutionary optimizations were performed for the fully bilaterally decoupled controller, and the decoupled controller with global sensing. Each optimization ran for approximately 3.5 hours on a 1 GHz Pentium III PC. Thus, the results presented below are from approximately 40 hours of data collection.
3.5.1
Complete Bilateral Coupling 25
20
15
10
5
0
−10
−5
0
5
10
Figure 3-5: Trajectories of the most successful bipeds in each of the six experiments. (Some of the trajectories overlapped.) 35
In the experiments with complete bilateral decoupling, five out of six runs evolved stable gait cycles. The history of the best fitness in each of the experiments is shown in Figure 3-4. The final fitness in all the experiments fell in a small range approximately between 7 and 10 ul. The gait was very similar in all these cases. An almost straight legged walking evolved with very small step size, using the rocking motion afforded by the cylindrical feet for ground clearance. The gait was very stable; although the controller was evolved to function for only 2000 time steps of the dynamic simulation, even after 5000 time steps most of the bipeds continued to walk, indicating that they had achieved a stable limit cycle. However, the trajectories often veered off sideways as shown in Fig 3-5. Out of the five successful walkers, two had trajectories which twisted too much and therefore were prematurely terminated, with a low fitness. The three others, also had trajectories which veered off the straight line path to a certain extent although not enough to be terminated. The average fitness therefore after 5000 time steps of all the experiments was 15.6, although the best was 24.34.
Figure 3-6: Physical Structure of the biped which achieved the highest fitness with complete bilateral decoupling. 4
3
2
1
0
−1
−2
−3
0
5
10
15
20
25
Figure 3-7: Foot placement history of the biped with highest fitness. Each footprint is represented by a . The best agent evolved in this set of experiments is shown in Figure 3-6. It had a lower leg width of 0.056 ul, an upper leg width of 0.097 ul, and a waist width of 0.62 ul. It used 36
Figure 3-8: Footsteps of the biped with highest fitness. Dark lines indicate that the foot is in contact with the ground. The length of the line indicates the duration of time for which the foot was in contact with the ground in one step. initial joint angle positions of [-0.57, 0.50, 0.62, 0.21, 0.10, 0.93] for the left knee, left hip roll, left hip pitch, right hip pitch, right hip roll and right knee joints respectively. Figure 3-7 shows this biped’s foot placement history. It can be seen that its trajectory is slightly curved, although there are long periods of stable alternating stepping. In Figure 3-8 the length of time during which each foot is in contact with the ground is plotted. It can be seen that its step duration is basically symmetric and alternates regularly although the left foot is in contact with the ground for slightly longer time intervals than the right foot. This asymmetry could be a consequence of the particular initial conditions set by evolutionary algorithm for this biped. Right Hip Roll
Left Hip Roll
0.6
0.6
0.4
0.4
0.2
0.2
0 −0.2
0 0.24
0.26 Left Hip Pitch
0.28
0.5
−0.2 0.23
0.24 0.25 Right Hip Pitch
0.26
0.5
0 0 −0.5 −1 0.12
0.14
0.16 0.18 Right Knee
0.2
0.22
0.04
−0.5 0.12
0.14
0.16 0.18 Left Knee
0.2
0.22
0.02 0.01
0.02
0 0 −0.02 −5.5
−0.01 −5
−4.5
−4
−3.5
−0.02 −6
−3 −3
−5
−4
−3 −3
x 10
x 10
Figure 3-9: Steady state phase plots of the joint angles of the biped with highest fitness. The x-axis plots and the y-axis . The measure of stability of this biped’s gait can be gauged in the steady state phase plots of the joint angles shown in Figure 3-9. The phase plots of the pitch and roll degrees of freedom at both hip joints show closed orbits with a high degree of regularity indicating a stable limit cycle. The knee phase plots also have closed orbits but they are more irregular and a closer look at the axis indicates that the orbits span a very limited range of positions and velocities close to zero, indicating that the knees did not move much at all. It basically seems that the controller actively locked the knee at , in order to use the straight-legged gait. Figure 3-10 shows the motor neuron activity for the three joints of the right leg. All three motor neurons show regular oscillatory patterns also indicative of a stable limit cycle. 37
Right Hip Roll MN
0.58 0.56 0.54 0.52
Right Hip Pitch MN
0.5 3400
3440
3460
3480
3500 time
3520
3540
3560
3580
3600
3420
3440
3460
3480
3500 time
3520
3540
3560
3580
3600
3420
3440
3460
0.7 0.6 0.5 0.4 3400
Right Knee MN
3420
−0.1 −0.2 −0.3 −0.4 3400
3480
3500 time
3520
3540
3560
3580
3600
Figure 3-10: Motor Neuron Activations of the Right Hip Pitch, Hip Roll and Knee joints, of the biped with highest fitness. No Coupling, Global Sensing 10
9
8
7
Best Fitness
6
5
4
3
2
1
0
0
50
100
150 Generations
200
250
300
Figure 3-11: Best fitness in each generation, for the six evolutionary optimization experiments on bilateral decoupling with global sensing.
3.5.2
Bilateral Decoupling with Global Sensing
In this set of experiments, the fitness graphs after 300 generations were similar to the case of complete bilateral decoupling as seen in Figure 3-11. However, on running each of the bipeds for 5000 time steps it was found that all of them had evolved stable walking. Moreover, the bipeds had achieved a high degree of directional control so that most of the trajectories were more or less straight, as seen in Figure 3-12. The gaits evolved were quite similar to those observed in the previous set of experiments: straight legged walking with small step size and rocking for ground clearance. However, since the trajectories were all straight, none of the runs had to be prematurely terminated. This yielded a high average fitness of 19.14, with a best fitness of 25.1. The best agent evolved in this set of experiments, had lower leg width 0.057 ul, upper leg width 0.1 ul, and waist width 0.13 ul as shown in Figure 3-13. It used initial joint angle positions of 0.00, -0.03, 0.62, -0.78, 0.61, and 0.00 for the left knee, left hip roll, left hip pitch, right hip pitch, right hip roll and right knee joints respectively. Figure 3-14 shows the
38
25
20
15
10
5
0
−10
−5
0
5
10
Figure 3-12: Trajectories of the most successful bipeds in each of the six experiments. (Some of the trajectories overlapped.)
Figure 3-13: Physical Structure of the biped which achieved the highest fitness with bilateral decoupling and global sensing. bipeds foot placement history over 5000 time steps. It can be seen here that after a transient phase where the foot placement is irregular the biped settles down into a stable straight line trajectory. Figure 3-15 shows the corresponding foot step durations of the biped during a brief time interval. As in the previous case the foot steps are regularly alternating in time. It is interesting that once again there is a slight bias towards the left foot being on the ground longer than the right. However, in this case the difference is smaller. The stability of the limit cycle is once again quite high for all the joints, as observed in Figure 3-16. The stability of the hip pitch and roll limit cycles is comparable to the case of complete bilateral decoupling. The knee limit cycle is more stable in this case. The motor neuron outputs shown in Figure 3-17 have a more rounded shape compared to those of the biped with complete bilateral decoupling. It seems that the global sensing allowed the controller to improve the smoothness of the motor neuron outputs and increase the stability of the gait.
39
4
3
2
1
0
−1
−2
−3
0
5
10
15
20
25
30
Figure 3-14: Foot placement history of the biped with the highest fitness. Each footprint is represented by a .
Figure 3-15: Footsteps of the biped with the highest fitness. Dark lines indicate that the foot is in contact with the ground. The length of the line indicates the duration of time for which the foot was in contact with the ground in one step.
3.6
Discussion
The results showed that a bilaterally decoupled neural controller could lead to synchronized motions of the right and left legs to produce to stable locomotion in a 6 DOF biped, using mechanical coupling. Furthermore, the directionality and stability of the gait could be improved without the need for explicit connections, only using global sensing. The question arises of whether such a neural architecture could be biologically plausible for human locomotion. It is certainly more plausible than an architecture which was found to be equally successful in previous experiments (whose results could not be presented here due to space constraints) in which two identical neural rhythm generators each received sensory inputs from sensors on both left and right sides. The current network achieves almost identical performance with much fewer afferent connections. However, the question of whether such a neural architecture is more biologically plausible than a controller with explicit connections in the neural controller, is more difficult to answer. As all the oscillatory units are believed to be located in the spinal cord, it is likely to be relatively cheap for the system to have connections between them if it leads to greatly increased stability and robustness. It would be necessary to perform further tests of robustness on decoupled controllers to determine whether bilateral coordination through mechanical coupling and global sensing is equivalent, inferior to, or superior to having explicit neural connections in the controller. Another issue is that in the experiments, the bipeds have cylindrical feet oriented parallel to the sagittal plane, enabling them to exploit the rocking motion for foot clearance. Although it would be desirable to have a more human-like gait, the straight legged walking does not in itself reduce the biological plausibility of the neural architecture, as the anti40
Right Hip Roll
Left Hip Roll
0.4
0.4
0.2
0.2
0
0
−0.2
−0.2
−0.4 −0.12
−0.11
−0.1 −0.09 Left Hip Pitch
−0.08
−0.4 −0.12
−0.07
0.5
0.5
0
0
−0.5
−0.5
−1 0.1
0.12
0.14 0.16 Right Knee
0.18
−1 0.1
0.2
0.02
0.02
0
0
−0.02
−0.02
−0.04 −6
−5
−4
−0.04 −6
−3 −3
−0.11
−0.1 −0.09 −0.08 Right Hip Pitch
0.12
0.14 0.16 Left Knee
−5
−4
−0.07
0.18
0.2
−3 −3
x 10
x 10
Figure 3-16: Steady state phase plots of the joint angles of the biped with highest fitness. The x-axis plots and the y-axis phase relationships already sensed for the hip movements through mechanical coupling could be propagated via ipsilateral connections to the knees and ankles. However, in these experiments the biped has 6 DOF and rigid ankles, as compared to 12 DOF or more in the human lower body. Also, the motoneuron outputs are used to control desired angle and not muscle force. Thus, it is a highly simplified biorobotic platform. To make a stronger bid for biological plausibility the results will have to be extended to a more anthroform simulation in future investigations.
3.7
Conclusions
In this paper it is shown that a bilaterally decoupled neural controller, consisting of two identical neural rhythm generators which each receive sensory inputs from one side of the body, is capable of producing stable locomotion in a 6 DOF biped robot in simulation. The bilateral coordination is achieved via sensing of the mechanical coupling of the legs through the body, to achieve stable straight-legged locomotion. Directional control can be improved only using global sensing. These results are in contrast to previous studies which have suggested the need for explicit contralateral connections in the neural control of biped locomotion.
41
Right Hip Roll MN
−0.16 −0.18 −0.2 −0.22 −0.24
Right Hip Pitch MN
3220
3260
3280
3300 time
3320
3340
3360
3380
3400
3360
3380
3400
3360
3380
3400
0.7 0.6 0.5 0.4
Right Knee MN
3240
3220
3240
3260
3280
3300 time
3320
3340
−0.1 −0.2 −0.3 −0.4
3220
3240
3260
3280
3300 time
3320
3340
Figure 3-17: Motor Neuron Activations of the Right Hip Pitch, Hip Roll and Knee joints of the best biped with highest fitness.
42
Chapter 4 Computational Role of Morphology This chapter is very different in nature from the other chapters. Whereas the other chapters adhere to the conventional format of scientific publication, where experiments are performed and conclusions are drawn based on the results, this chapter is much more exploratory in nature. The main focus of the chapter is a thought experiment, followed by sections which discuss new facets of the interaction between morphology and neural control in light of the experiment. The paper is a collection of ideas which due to being in the nascent stages still has some conceptual gaps in between them. Nonetheless, it is felt that the paper has something important to add to the understanding of the interaction between morphology and control in biped locomotion, which is why it is included here.
Morphology and Computation1 The body of a situated agent plays an important role in adaptive behavior as it enables sensory motor interactions with the environment which can give rise to emergent intelligent behavior. Using the physical dynamics of the body, the agent can perform behaviors with much simpler control than would otherwise be required for the same task. The physical structure of the body, or the morphology, determines its possible set of sensory motor interactions as well as its dynamics, and as a consequence, determines the complexity of the control required. This intrinsic relationship between the morphology and required control has been identified as the morphology and control tradeoff [83]. What has not been previously considered, however, is that the controller is a computational entity whereas the morphology a physical entity, and if a tradeoff exists between them, it may be possible that the morphology can subsume a computational role. It is this computational contribution which may then reduce the computational demands on the controller, giving rise to the common phenomena of the morphology and control tradeoff. This paper introduces this idea and proves that it is indeed possible for the morphology to perform computation which can reduce the computational demands on the controller. 1
Paul, C. (2004) ”Morphology and Computation”. In Proc. 8th Int. Conf. on Simulation of Adaptive Behavior, MIT Press, Los Angeles, CA, USA.
43
4.1
Introduction
It is known in the study of adaptive behavior that the morphology of an agent’s body can be used to achieve simplified control. Brooks showed that embodiment would enable the use of simplified controllers requiring little or no representation [8]. Pfeifer has similarly discussed the importance of considering the design of the morphology of a robot, to reduce or simplify the control required and has named this relationship the morphology and control tradeoff [83]. Braitenberg has shown through a series of simple vehicle designs that the control of behaviors such as obstacle avoidance and light following can be achieved using simple sensory-motor control if the morphology, in particular the placement of sensors and actuators is appropriate [6]. In a more biologically inspired system, Cruse [17] gives an example of how mechanics of the body are used in biology, to simplify the coordination of insect legs. Although these studies have highlighted the reduced computational complexity of control as a result of the dynamics of the body, they have not considered that this may be the case because in effect the morphology performs some of the “computation” that the controller would otherwise have to perform. This paper shows on the basis of the thought experiments in Sections 2 and 3 that it is indeed possible for the morphology to perform computation which in effect reduces the computational requirements on the controller.
4.2
The XOR Robot: a thought experiment
Consider a robot which is controlled by perceptron networks. The perceptron is a wellknown neural network, which consists of a single output neuron, with multiple inputs, adjustable synaptic weights and a threshold function. The perceptron convergence theorem, proved by Rosenblatt showed that the perceptron could be used to compute any function, as long as it is linearly separable [40]. This means that the perceptron could be used to compute functions such as AND and OR, for example, but not functions such as XOR or XNOR. Consider now that there are two perceptron networks, with inputs A and B. The first one performs an OR function, and the second an AND function. The networks are connected , to actuated parts of a simple robot body (Fig. 4-1). The first network is connected to which is a wheel at the center of the base of the robot which causes forward motion. The second network is connected to , which lifts the wheel off the ground. Now when A and B are both off, both networks output 0, which means that the wheel is on the ground but does not move, so the robot is stationary. When only A is active, the AND network outputs a 0, so the wheel remains on the ground. The OR network outputs a 1, and makes the wheel turn, thus causing forward motion. When B is 1, the situation is the same, the AND network outputs a 0 and the OR outputs a 1, causing forward motion. However, when A and B are both on, then the OR network causes to move, but the AND network causes the wheel to lift off so that it no longer touches the ground so the vehicle is once again stationary. The following table summarizes the behavior of the robot in these four conditions.
44
~ F F T T
F T F T
Behavior stationary moving moving stationary
This table looks like the truth table of the XOR function. How is this possible?
M2
M1
AND
A
OR
B
A
B
Figure 4-1: The XOR Robot: This robot has one wheel, with two actuated degrees of freedom. The motor M is responsible for turning the wheel so that the robot moves forward. The motor M is responsible for lifting the wheel off the ground. Each motor is controlled by a separate perceptron network, which takes as inputs A and B. M is controlled by a network which computes A OR B, and M by a network which computes A AND B. Using only these controllers, the robot is able to display the XOR function in its behavior. The explanation is that the robot’s behavior is not simply determined by the output of the neural networks. It is also affected by the interaction between the actuated components of the body. The body through its mechanical structure provides some additional computational ability, which allows the XOR function to be displayed. In this case, the body provides the computational equivalent of a second layer of neural processing, in which it performs a NOT on its first input, followed by an AND, as shown in Figure 4-2. The func tion can be written as . This clearly shows that through its morphology, a = robot body can perform a quantifiable computation which reduces the computation required in the controller.
4.3
The OR Robot
It can be shown that such computational ability is by no means an isolated occurrance. In fact, various simple morphologies can give rise to computation. In order to show this 45
U M2
M1 AND
A
OR
B
A
B
Figure 4-2: Computational structure equivalent to the XOR Robot: The body of the XOR robot acts as if it is performing the computational function M AND B
M1
M2
Figure 4-3: The OR Robot: This robot has two wheels both of which turn in the same direction when actuated. Each wheel is actuated by one motor which is responsible for turning the wheel. The wheels are also capable of being driven passively. through a second robot example the OR robot is presented. The OR Robot (Fig. 4-3) is a simple mechanical structure, with a rectangular body and two wheels which are aligned along central longitudinal axis of the body, one behind the other. Each wheel has a motor, which when turned on, rotates the wheel so that the body is propelled in the forward direction. When the motor is off, the wheel can also be passively driven. In this robot, when both motors are off, the robot is stationary. When only is active, one wheel turns and passively drives the other wheel so the robot moves forward. Similarly, when only is active the robot, the robot also moves forward. Finally, if and are both active, both wheels turn, and so the robot also moves forward. The behavior of the robot in these four conditions is summarized in the table below.
46
S
S
B
I
H
H
H
H
H
H
H
H
M1
M2
A
Figure 4-4: Conventional neural network architecture for Vaccuum Cleaning Robot: The robot is controlled by a neural network which has an input layer with sensory input nodes, labelled S, input node I and bias node B. There are two hidden layers, each with 4 hidden neurons each, labelled H. The final layer is the output layer which has two output nodes to send motor commands to the robot.
X F F T T
F T F T
Behavior stationary moving moving moving
It is clear that this table is the truth table of the the familiar OR operator from Boolean X and , the computalogic, and that when a controller sends commands to the motors tional function performed by the controller is augmented by a final OR operation. Utilizing this to achieve the desired overall computation can reduce the computational requirement of the controller by one logical function.
4.4
Explicit Computation
Although the computational abilities of the robot morphologies described in Sections 2 and 3 are relatively small they serve as an important proof of concept, that morphology can perform computation which can reduce the computational requirements of the controller. 47
At first glance, it may be argued that this computation simply exists in the eyes of the observer, and is not a real computation. In this section, adopting the commonsense definition that a computation is real if it can be used as a computation, it is shown that the computation performed by the morphology of the XOR robot is in fact real.
B
I
S
S
H
H
M1
M2
H
H
A
M1
M2
A
Figure 4-5: Morphological computation used in the neural control of the Vaccuum Cleaning Robot: The robot is controlled by a neural network which has an input layer with sensory input nodes, labelled S, input node I and bias node B. There are two hidden layers. The first hidden layer has 2 hidden nodes, represented by H, and two nodes which convey motor commands to the robot, represented by M and M . The second hidden layer also has two hidden nodes, labelled H, and a node which conveys the output of the robot accelerometer A. The final layer is the output layer which has two output nodes which send motor commands to the robot. Consider this example. The body of the XOR Robot is now to be used as the basis of a real world robotic application such as vaccuum cleaning. It is enhanced with cameras to sense the environment, a microprocessor for computational processing and behavior arbitration, a binary velocity sensor and of course a vaccuum unit. Its task is to survey its initial environment and decide whether to vaccuum or not. If the floor does not look particularly dirty, it can wait. If the floor is dirty, it can decide to start vaccuuming down the long straight corridor. Such behavior arbitration requires a certain amount of processing. A neural network based controller could be used for the task. This may look like the network shown in Fig. 4-4. This is a standard neural network architecture with an input layer, two hidden layers 48
and an output layer. The input layer receives inputs from the robots visual sensors S, a higher level signal I, and a bias node B. The inputs are processed through the three layers of the network and result in output motor commands issued to the robot. Each individual node in such a network is a computational sub-unit of processing. Now since the robot is equipped with a velocity sensor it is possible to measure the behavioral outcome of the robot as a result of motor actuation. Making the simplifying assumptions that the robot cannot overcome inertia and the force of friction when the wheel is not in contact with the ground or not actuated, and that when moving, the robot comes to an immediate halt when the motors are switched off due to gear resistance and friction. Thus, the output of the velocity sensor directly depends on whether the wheel is in contact with the ground and actuated and in essence conveys the output of the computation performed by the body. Using this sensor, the robot body itself could then be used as a computational subunit within the controller. For example, we could use it to replace a node in the network, as shown in Fig. 5. As can be seen Fig. 5 is very similar to Fig. 4 except that in the second hidden layer of the network one node has been removed and another has been replaced by the robot body such that it receives inputs from two neurons of the previous hidden layer which actuate the motors, and sends binary outputs measured by the velocity sensor, A. Removal of a neuron obviously does not diminish the computational nature of the network, although it may alter the function. Similarly, replacing a hidden neuron with the robot body, simply changes the function of that computational sub-unit from a standard threshold function on the weighted sum of the inputs, to X\z . It does not invalidate the controller as a computational entity itself and therefore demonstrates that the transformation performed by the morphology can be used as a traditional computation, and is therefore real. This example is of course a toy example. Performing the XOR function on a processor would be trivial from the point of view of processing time and energy costs compared to having to move the body. However, in a more complex morphology where a simple motion may perform the equivalent of an extremely complex computation, it may be useful to utilize this ability.
4.5
Complex Morphology
In the above example, the morphology of the XOR robot performed a simple binary function. served as the basis for the Vaccuum Cleaning Robot. The computation performed by the body of the XOR robot was easy to recognize as a common binary function, and so it could be shown that starting from a morphology which performed a simple computation, a controller architecture could be designed to utilize it. In robots with more complex morphologies such as manipulators or legged robots, which have rotary or prismatic actuated joints and joint angle sensors for sensing, the computation performed is a complex analog function determined by the dynamics of the body. Yet, it is nonetheless a computational function, in the sense of defining a mapping from motor commands as inputs to the physical consequences measured by the sensors as outputs. Consider for example a 2 DOF robot manipulator with rotary joints and joint angle 49
B
I
H
H
H
M1
M2
S
S
H
S2 S1
Figure 4-6: Control of a robot manipulator: A 2 DOF manipulator is is controlled by a neural network with sensory feedback. The network has an input layer with 2 sensory input nodes, labelled S, an input node I, and a bias node B. The sensory nodes recieve proprioceptive inputs from the manipulator on joint angles. There is one hidden layer with four hidden nodes represented by H. The output layer has two output nodes which send motor commands to the two actuators of the robot, M and M sensors, which is controlled by a neural network with sensory feedback (Fig. 4-6). For simplicity we assume that input is 0 and the weights of the network are constant. The controller is designed such that it reads in sensory input from the joint angle sensors, processes the information through the network, sends motor commands, and only then reads in the next value of sensory input for the next iteration of neural processing. It is clear to see that in each such iteration there are two transformations. One is the transformation of the sensory inputs to motor outputs through the neural network. The second is the transformation of motor outputs to sensory inputs through the forward dynamics of the manipulator. Thus, the entire system forms a single computational entity. Viewing the system from this perspective, it is clear how a morphology and control trade-off can exist in an embodied agent with sensorimotor control. Changing the physical characteristics of the robot, including even simple parameters such as link length or mass distribution, will affect the dynamics of the manipulator. This will, in effect, change the computational relationship between the motor commands and its physical consequences measured by the sensors. In some cases this change may be for the worse, but in others it may be for the better with regards to the task, and reduce the requirements on control.
50
4.6 4.6.1
Discussion Morphological Computation
The thought experiments of Sections 2 and 3 demonstrate that it is theoretically possible for the morphology of an agent to perform a computational function. Such computation which can be latently performed by the morphology can be called morphological computation. The knowledge that physical structure can perform such computation is as old as Babbage’s design for his Analytical Engine, a massive brass, steam-powered mechanical computer, from 18372 . But as computation has grown more siliconized in the 20th century, a conceptual divide has arisen between the computational and physical hardware in robotics and AI. The body is viewed as an effector for the computations performed by the control hardware. This has occluded understanding the basis of the morphology and control trade-off. The main aim of the previous sections has been to remove this occlusion and show that computational properties can arise even in very simple morphologies reducing the requirements on the controller.
4.6.2
Explicit morphological computation
The Vaccuum Cleaning Robot demonstrates that when a robot with latent morphological computation is augmented with a sensor which can sense the behavioral consequences, it makes the computational function defined by the morphology explicit, such that it can be used as a standard computational sub-unit, at any stage of processing. The fact that this is possible shows that morphological computation is not simply a way of viewing the transformation of motor commands to physical behavior as a computational process, but that it is a real computation as is traditionally understood, and can be used accordingly.
4.6.3
Using computational and motor functions
The Vaccuum Cleaning Robot also shows that in a system where appropriate sensing enables the explicit use of morpohological computation, a controller can exploit both the computational and motor functions. This can be achieved by separating the two functions in time, and alternating between them.
4.6.4
Duality
In engineering, it is traditional to think of a single component of a system performing a single function. Thus, in most cases in human engineered technology each component usually has a single function, and if it has more than one function, it is usually used to perform only one of them at a time. This modular approach to design is the basis of human engineering, which leads to a tacit belief in the idea that natural systems may also be designed according to these principles. However, in the case of an intelligent embodied agent, this does not 2
Babbage, C. (1864) Passages from the Life of a Philosopher, Chapter VIII
51
need to be the case. Although, it is difficult for human engineers to conceptualize, it is possible for a single action of an agent to serve both motor and computational functions at the same time. In fact, many physical action performed by an embodied agent have the inherent possibility for duality of function. In order to illustrate the concept, and show that it is relevant for natural intelligence, a simple analogy is made to human behavior. Consider the case when there are a large number of coins on a desk which must be counted (more than can be counted through a cursory visual inspection) and they must be put away in a drawer under the desk. Notice that there are two components to the task requirements. One is the computational task of counting the coins, which involves the development of an abstract representation (number) based on the environmental condition. The second is a physical task of relocating the coins from the desktop to the drawer. There are multiple ways of accomplishing this, but one common approach to the task that humans use, is to slide the coins off the desktop into the drawer one by one, counting each one in the process. In this simple behavior, the action of sliding the coin into the drawer has duality of function. One function is computational: it serves to separate the coin which has already been counted from the set of coins which has not been counted. The second function is physical: relocating the coins from the desktop to the drawer. These two functions are seamlessly integrated into one action, without any conscious forethought. The example demonstrates that if the opportunity arises to utilize duality of function, natural intelligence has been designed to be able to exploit it. The Vaccuum Cleaning Robot, however, was an example of “human design” in that the computational role of the body was separated from the physical role in time. This had the advantage that it highlighted the use of the computational function of the body, by separating it from its physical role. However, theoretically, no such separation is necessary. It should be possible to design systems which use their physical structure for both physical and computational purposes at the same time.
4.6.5
Effect of the environment
The environment plays an important role in morphological computation. In the thought experiments of Sections 2 and 3, the systems were embedded in a single environment. However, when the environment of the system changes, the functions describing the system will also change. This is very clear to see in the examples presented. Consider the case when the OR robot is placed on a downhill incline. Now, the robot moves forward regardless of whether the actuators are active or not. The computational function of the structure changed. Thus, the ecological niche is crucial for the particular characteristics of the morphological computation which arise.
4.6.6
Morphology and Control
It has been understood in the study of adaptive behavior that embodiment can lead to simplified control. However, the reasons for this have not been clear. It has been known that interaction of loosely coupled sensorimotor processes with the dynamics of the body gives rise to emergent behaviors which are not explicitly represented in the controller [7][66][83]. However, the mechanism of emergence itself has not been further investigated. By showing 52
here that the body can perform a computational role, the basis of such emergence can begin to be understood. In some cases at least it is can be understood that the body provides the “computational glue” between loosely coupled sensorimotor processes. The fact that such computation can be so simply achieved suggests that it is not simply a rare phenomena, but possibly a pervasive characteristic of physical structures. Further scientific investigation is necessary in this area. There also is a long way to go in understanding morphological computation and its relationship to control. Particularly it needs to be understood what characteristics of the morphology affect control requirements, and how. It is likely that such understanding will require the development of new analytical techniques to identify and quantify the computational contributions of a morphology. If such an understanding can be achieved, however, it will open a new realm of possibilities for designing robots which incorporate “intelligence” in their bodies. By utilizing smart morphologies which are known to optimize dynamics for control, more robust and adaptive agents will be able to be designed.
4.7
Conclusion
This paper introduced the concept of morphological computation, computation performed by the mechanical structure of a robot body. To show the existence of such computation, an example of a robot controlled by perceptron networks was given, which utilizing the structure of its morphology was able to display the XOR function in its behavior. The XOR function being linearly inseparable would have been impossible to achieve with only a perceptron network, and thus proves that the morphology can perform a computational role. This was followed by the description of a vaccuum cleaning robot which used the computation performed by its body in its own neural control. The example demonstrated that the computation performed by the body is both explicit and real. Furthermore, it explicitly illustrated a situation where the morphology replaced part of the computational control. The illustration sheds new light on how smart morphologies could reduce control. Future work will focus on developing a theoretical framework for morphological computation which will provide analytical methods for studying the computational role of a robot body. This could be used to guide the design of smart mechanical structures, which will be able to exploit the morphology to reduce control.
ADDENDUM As the concept of morphological computation is still in its nascent stages, it should be admitted that there are still conceptual gaps in the paper. For example, although sections 4.4 on Explicit Computation and 4.5 on Complex Morphology are both internally coherent, there is a very large conceptual leap between them made, for the moment, only on the wings 53
of intuition. The same is the case of Section 4.6.4 on Duality. The author is fully aware that further theoretical developments will need to be made in order to bridge these gaps. For the moment, however, it should be clarified how the concept of morphological computation is related to the morphology and control trade-off. The principle of the morphology and control trade-off generally indicates that an intrinsic relationship exists between the morphology of an agent and its required control. Depending on the morphological characteristics, the control requirements of the agent may either be larg or small. One main interest of the author was to understand the mechanism underlying this trade-off. Why does it exist? Following this line of enquiry led the author to see that the morphology and control trade-off is not simply one effect, but arises as a result of multiple underlying mechanisms which give the appearance of a single effect. One underlying cause is simply that some morphologies are more stable than others. It is easy to see, for example, that a walking robot with large flat feet will be easier to control than one with point contacts. But another completely separate underlying cause is morphological computation. The robot body can in fact perform some of the computation that the controller would otherwise be required to perform, thereby reducing the control requirements. Thus, morphological computation is an underlying cause of the morphology and control trade-off, which may play an active role in biped locomotion.
54
Part II Upper Body
55
Chapter 5 The STUMPY Hopping Robot Design and Control of an Inverted Pendulum Driven Hopping Robot1 In this paper a new kind of hopping robot has been designed which uses inverse pendulum dynamics to induce bipedal hopping gaits. Its mechanical structure consists of a rigid inverted T-shape mounted on four compliant feet. An upright “T” structure is connected to this by a rotary joint. The horizontal beam of the upright “T” is connected to the vertical beam by a second rotary joint. Using this two degree of freedom mechanical structure, with simple reactive control, the robot is able to perform hopping, walking and running gaits. During walking, it is experimentally shown that the robot can move in a straight line, reverse direction and control its turning radius. The results show that such a simple but versatile robot displays stable locomotion and can be viable for practical applications on uneven terrain.
5.1
Introduction
The design and implementation of the STUMPY II2 hopping robot is an exploration of a novel morphology for locomotion, with an inverted pendulum inducing rhythmic hopping and a transverse rotational degree of freedom for direction control. Its unique structure and dynamics are capable of producing both biped-like and quadruped-like gaits. In addition it can also display some effective non-biomimetic gaits. Such use of pendulum dynamics in movement has been only partially explored. Hayashi et al [39] have designed a pendulum-type jumping machine which uses inverted pendulums as swinging arms to propel the robot to jump. This machine was capable of jumping up stairs, but not of regulating its movement direction or velocity. In another interesting example, Ioi et al [48] applied pendulum dynamics to the problem of wheeled locomotion and designed a robot comprised of two big parallel wheels, with a pendulum hanging between them. This robot was able to roll up slopes and control forward velocity and turning. 1
Paul, C., Iida, F., and R. Dravid (2002) ”Design and Control of a Pendulum Driven Hopping Robot ”, in IEEE/RSJ International Conference on Intelligent Robots and Systems Lausanne, Switzerland. (All three authors have equal contributions to this paper.) 2 Videos available at http://www.ifi.unizh.ch/ailab/robots/Stumpy/
56
However, the use of pendulum dynamics to drive legged locomotion has only been rarely considered.3 4 The control of gait and balance in hopping robots has been widely studied by Raibert and his colleagues, for one-legged, two-legged and four-legged hopping robots, in two and three dimensions [95] [94] [97] [96]. Such robots have been shown to successfully produce bounding gaits, through control of hopping height and forward velocity. They have also been able to perform somersaults [86]. These robots have long narrow legs each with a single spring-loaded prismatic joint. The legs are attached to the body mass, by a roll and (for 3D motion) a pitch degree of freedom, by which they can influence foot placement. Due to this mechanical structure, these robots are statically unstable, and therefore must continue to hop in order to stabilize their body. For practical applications this presents a considerable limitation. In the STUMPY robot, the four legs also consist of spring loaded prismatic joints, but are much shorter. Also, they do not have any other degrees of freedom with respect to the robot’s upper body, and control of foot placement is accomplished by the upper body. This has the advantage that the structure is statically stable, while allowing for dynamically stable locomotion. This enables the robot to smoothly transition between standing still, walking, running, and coming to a stop again, which is more practical for real world applications. The dynamic stability of the STUMPY robot is comparable to that of the monopod hopping robot developed by Ringrose [100]. The monopod was able to achieve self-stabilizing running, without any sensors or active control, by simply moving the single actuator of the robot, through a fixed repetitive cycle. Due to the interaction of this simple control with a carefully designed self-stabilizing mechanical structure, the robot was able to correct its posture despite the effects of destabilizing forces. According to Murphy [75], selfstabilizing posture of the body can be achieved in a two legged robot during bounding gait, if the normalized moment of inertia of the mechanical structure is less than 1. Since the interaction between controller and the mechanical structure of the STUMPY robot fulfills this condition, it displays a similar self-stabilizing property. Thus, it is able to perform locomotion with high tolerance to environmental disturbances. The mechanical structure of the robot has been designed according to the principles of cheap design and ecological balance described by Pfeifer et al [83]. Thus, only the minimum number of sensor and actuators that are necessary to perform the task have been used. This is similar to the design of the Scout robot, built by Buehler et al [11], which uses a small number of actuators for quadruped locomotion. The design of STUMPY has also incorporated the use of mass distribution in control, an optimization method described in Chapter 2. The controller for the robot is designed according to the principles of behavior based control [9]. The following section, Section 2, describes the design of the STUMPY robot. The behavior of the robot is then mathematically modelled in Section 3. Then the control of 3
changed from has not been previously considered in the original text. At the time, the authors were unaware of the work of Paramonov et al. [77], in which humanoid robots were developed to walk using swinging pendulums. 4 This is conceptually different from using inverse pendulum dynamics in ZMP-based balance control, which is common in legged robots
57
straight line movement, direction, and turning radius are developed in Section 4. These controllers are tested on the robot, and data is presented from real world experiments. In Section 5, a short discussion of the robots performance on these tasks follows, with implications for future work. Section 6 ends with conclusions.
5.2
Robot Mechanical Structure
The robot (Fig. 5-1) was designed through the development of three prototypes. The STUMPY II robot’s lower body is made of an inverted “T” mounted on wide springy feet. The upper body is an upright “T” connected to the lower body by a rotary joint providing one degree of freedom in the frontal plane (see Fig. 1a, 1b). This enables the upper body to act as an inverted pendulum. For simplicity in nomenclature, we call this the “waist” joint. The horizontal beam of the upright “T”, is weighted on the ends to increase its moment of inertia. It is connected to the vertical beam by a second rotary joint, providing one rotational degree of freedom, in the plane normal to the vertical beam of the upper “T”. This joint is labeled the “shoulder” joint. STUMPY’s vertical axis is made of aluminum, while both its horizontal axes and feet are made of oak wood.
Figure 5-1: STUMPY Robot: photograph of the robot Fig. The total mass of the robot is approximately 1.9 kg. The mass and length parameters of the robot, as shown in Figure 5-2 are detailed in the Table 1 below.
58
Figure 5-2: Schematic diagram of the STUMPY Robot, with variables which are used in modelling and analysis. Parameter ,
¡ ¡ ¡
Description Value rest length of feet 10 cm length of base 15 cm length of lower vertical beam 21 cm length of the upper vertical beam 26 cm length of shoulder horizontal beam 41.5 cm mass of lower body 1.2 kg mass of upper body 0.43 kg mass on shoulder 0.12 kg
Table 1: Mass and length parameters of the robot mechanical structure The joints are actuated using Minimotor DC-Micromotors. The waist motor is a 3042 012C with a 43:1 gear reduction, and shoulder motor is a 2342 012C, also with 43:1 gear reduction. The joint angles are measured using rotary potentiometers. The control is performed via off-board motor control boards with a PIC16F877 microcontroller and a standard motor driver with PWM output.
5.3
Modelling and Analysis
The following variables are used in the analysis of the robot:
¢£%¤¦¥v£ ¤§ £ : origin of the world coordinate system 59
¨ ¢{¤¦¥U¤¦§v©ª¤¦« : origin of the robot, inclination in frontal plane ¬ ¤ ¬ : waist motor torque, shoulder motor torque « ¤« : joint angles of the robot *® : moment of inertia of the lower body
To analyse the behavior of the robot, the model is orthogonalized into models for the frontal and sagittal planes. In the frontal plane, the waist motor serves to accelerate the waist joint and induce hopping motion.
¬ °¯ ¡ ²« ± ´³ ¡ ¦µ ·¶¸¹{º « »r¼!«G½¾
(5.1)
We assume that in this motion, the effects of external forces are negligible, and that conservation of angular momentum holds. According to this the following relation can be stated:
®À« ¿ ®
¿ Á¯ ¡ «
(5.2)
where is the moment of inertia of the lower body. During walking, where at least one foot is always on the ground, the following relations arise:
¢ ¯ 6Â[ö « ¤§ ¯ 6¶\¸8¹ «
(5.3)
By substitution of Equation 3 into Equations 1 and 2, the following relations result:
*® ¢ ¿ *® ¿§ ¯
¯
º¬
Ķ\¸8¹ « ¾ º ¶¸¹ « ¾ ¼z¡ ¦µ ³
(5.4)
º¬ z ·¶¸¹ « ¾ º Âeö « ¾ ¼ ¡ ¦µ ³
(5.5)
In the sagittal plane, motion is induced by application of torque at the shoulder joint. The shoulder torque first accelerates the shoulder beam.
¬ ¯ ¡ « ±
(5.6)
¬KÅ ¼ [¬ Ç ¯ * ® e« ± È É ³ ¬eË
(5.7)
This motion causes two effects. One is a reaction torque, ¬KÅ ¯Æ³ ¬ , which must be produced to conserve the angular momentum due to the rotation of the shoulder beam. The second is an impulse torque produced due to the collision of the shoulder beam with the ¿ joint stop ¬[Ç . At any given angular velocity of the shoulder joint « , the reaction torque ¬KÅ ¿« will have the opposite sign as , and ¬eÇ will have the same sign. The rotation of the lower body in the plane normal to the reaction torque, «eÈÊÉ , caused by shoulder motion, is thus given as follows: where ¬eË is the friction torque produced by the coefficient of static friction between the ground and the feet.
60
5.4
Control
STUMPY is controlled to move in a unique way by actuating its waist joint, with a back and forth swinging motion. This motion of the upper body imparts angular momentum to the base which creates a rhythmic hopping motion. During hopping, each lateral pair of foot contacts which can be considered together as a “foot”, experiences two phases: stance phase, in which the foot is on the ground and flight phase, during which the foot is airborne. At low frequencies of the upper pendulum, one foot completes its flight phase and returns to stance phase, before the second foot initiates flight phase. As a result, there are two phases in the gait cycle: single support and double support. Therefore, we term this gait “walking”. At higher frequencies, one foot is still in flight phase when the next foot enters flight phase, so the two phases in the gait cycle are single support, and airborne. Therefore we term this gait “running”. At the transition frequency, there is only one phase: single support. If the “shoulder” joint is unused, the robot will hop in place. The shoulder joint can be used to control movement in the sagittal plane. When a lateral forward rotation of the shoulder joint occurs, the foot will acquire a slight angular momentum during flight phase, which serves to project the foot forward. Thus the shoulder joint can be used to control movement direction, forward velocity and turning rate. The control parameters are the frequency Ì´ , amplitude ÍÁ and setpoint «vÎ of the waist joint oscillation, the frequency ÌR , amplitude Í{ and setpoint « Î of the shoulder joint oscillation, and the phase difference, Ï , between Ì° and ÌÐ , when they are oscillating at the same frequency. The basic control for the waist motor is a simple reactive algorithm, which outputs a constant motor voltage until the joint angle sensor detects the maximum amplitude ÍÁ , and then reverses the direction. Thus, the waist motor oscillates approximately between ÍÁ and ³ÑÍÁ . The control of the shoulder joint is reactively coupled to the waist and functions in a similar way. The shoulder motor voltage remains constant, until the waist joint angle sensor detects that it has crossed the maximum amplitude ÍÁ . At this instant, the shoulder joint instantaneously reverses the motor torque. Due to this coupling, the waist and the shoulder joints are phase locked, such that the phase relationship Ï between the waist and the shoulder joint is constrained to be either ÒÓ or »%Ô ÒÓ . The hopping height of the robot mainly depends on the angular acceleration of the waist joint periodic motion. The maximum amplitude of the swing determines the region of stability of the structure during hopping. As in Raibert’s hopping robots, it has been experimentally found that the control of hopping height in this robot can be separated from the control problems of forward velocity, gait direction, and the radius of curvature of the robots turning trajectory. As the main focus was the control of movement direction, the hopping was tuned to a constant frequency, which produced a suitable hopping height and allowed for a wide range of motions. Controllers were then developed for straight walking, reversing direction and turning. These controllers were implemented and tested on the robot. Data on position and velocity of the robot during locomotion was collected using a CCD camera suspended from the ceiling above a 3.0 m x 2.0 m experimental arena, and a framegrabber which recorded the movement of the robot at 25 frames/sec. For each experiment, the robot was initially 61
positioned at the center of the image. It was equipped with four high-intensity LEDs, two on each side of the base and recorded in a darkened room. The camera image was then processed to identify the locations of the LEDs, from which the robots position and orientation was extracted and plotted once every second. The following sections describe the design and implementation of the controllers and present experimental data collected using this setup.
5.4.1
Tuning the Hopping Height
The hopping height and frequency were experimentally tuned by adjusting the motor voltage of the waist motor. It was found that if the motor voltage is too low, the angular acceleration is not fast enough to lift the feet off the ground for a significant amount of time. As, the motor voltage gets higher a stepping motion is induced but the frequency is not in tune with that of the mechanical structure. Within a small range of motor voltages, resonance with the mechanical structure produces a stable hopping pattern is produced with high dynamic stability. Informal tests show that even large disturbances caused by accidental human intervention, such as “stepping on the robot’s foot”, are corrected by its self-stabilizing nature. In this range, the robot produces a “walking” gait, where one foot is always on the ground, and therefore lends itself to good direction control.
5.4.2
Straight Walking
As described above, the shoulder joint is coupled to the waist joint angle, and reverses the motor voltage at the maximum amplitude values. Thus, the basic control is such that Õ × Ö , the motor voltage applied to the shoulder during clockwise rotation, and Õ"× Ø which is the motor voltage applied to the shoulder during counter-clockwise rotation are related by relationship, Õ × Ö ¯Ù³ Õ × Ø . In this situation, the forward momentum acquired by the left foot during flight phase, is equal to the forward momentum acquired by the right foot and, therefore, the alternating step sizes of the left and right foot are equal and the robot walks in a straight line. Results of the robot performing this behavior are shown in Figure 5-3. (The axes in this figure, and all other performance graphs are in metres).
5.4.3
Reversing Direction
It was found that the direction of the robot, that is whether it would move in the forwards or backwards direction, was only dependent on the phase parameter Ï . The following graph in Figure 5-4 shows how changing the phase from ÒÓ to »%Ô ÒGÓ , makes the robot move in the opposite direction from that in Figure 5-3.
5.4.4
Control of Turning Rate
The control of turning rate is achieved using an enhanced forward walking controller. In forward walking, the step lengths on the left and right sides are approximately equal. Using a similar controller, but by increasing the step length of one foot and shortening the other, 62
Figure 5-3: Forward walking, produced when Õ × Ö ¯
Figure 5-4: Backward walking, produced when Õ × Ö ¯
Õ ×Ø ¤ Ï ¯ÒGÓ
Õ× Ø ¤ ϯ
»%Ô Ò Ó
the robot can achieve turning. Thus, we focus on the turning rate induced by the difference between the clockwise and counter-clockwise voltages applied to the shoulder motor. The º ¿ turning rate, Ú , can be approximately given as Ú ¯ÆÛ Õ × Ö ³ Õ × Ø ¾ . Thus, for example, Ö Ü Õ × Ø , while the motor rotates clockwise, the foot in flight phase takes in the case of Õ × a large step, but while the motor rotates counter-clockwise, the other foot takes a smaller step, causing the robot to turn at a constant rate. The control of movement in the forward or backward direction, discussed in the previous subsection, is also independent of turning rate. Various combinations of Õ × Ö and Õ × Ö have been tested, to exhibit different turning rates in both forward and backward walking. Figure 5-5 shows a small turning rate produced Ö Ý Õ × Ø . Figure 5-6 also shows the performance with the same by the motor control of Õ ×Ù turning rate, but in the other direction, that is where Õ × Ö Ü Õ × Ø . Figures 5-7 and 5-8 show the same behaviors but in the reverse direction. Then, larger turning rates are produced, when the difference between Õ × Ö and Õ × Ö is greatly increased, which effectively produces turning in place, as shown in Figure 5-9 and 5-10.
63
Ö Ý Figure 5-5: Turning left, with a small turning rate, produced when Õ ×!
Õ"× Ø .
Ö Ü Õ ×Ø . Figure 5-6: Turning right, with a small turning rate, produced when Õ ×Þ
5.5
Discussion
The controllers tested for control of forward motion, change of direction and turning rate, as described in the previous sections, achieved considerable success in displaying the desired behavior in real world experiments. The results showed that the preliminary controller concepts developed, were sufficient to fulfil the requirements of the tasks. However, the controllers were all open loop with respect to movement direction and velocity, and were thus sensitive to slight changes in mechanical biases in the robot, and environmental irregularities such as frictional differences on the ground and slight ups and downs of the terrain. To account for these uncertainties closed-loop control using sensor feedback must be added to these controllers. The selection of appropriate sensors and their use in a closed-loop control algorithm, are topics of further investigation. In addition, many other interesting issues with respect to the performance and control of this robot remain to be investigated. One such topic is the self-stabilizing property of the robot. In this paper, this property was exploited to achieve high dynamic stability during locomotion. However, it would be interesting to focus on this aspect more closely, and explicitly analyse the domain of attraction and its relationship to mechanical and control 64
Figure 5-7: Going backwards and turning right, with a small turning rate, produced when Õ ×z Ö Ü Õ × Ø . Simply by changing Ï to »%Ô ÒÓ , the robot walks backward, as it slowly turns right.
Figure 5-8: Going backwards and turning left, with a small turning rate, produced when Õ × Ö Ý Õ× Ø . parameters. Another topic of potential interest would be the accurate control of forward velocity. Finally, what would also be of interest is the investigation of different gait patterns. The STUMPY mechanical structure is capable of moving in a variety of different ways, one of which (walking) has been investigated in this work. The exploration of some of the other dominant modes of locomotion such as running, quadruped-like lateral bounding, and “diagonal” gaits, will be exciting topics of further investigation, which will further exploit the versatility of the robot.
5.6
Conclusions
In this paper a new kind of hopping robot with two feet, and no legs, has been presented. It consists of an inverted T-shaped base, connected to a T-shaped pendulum by a rotary joint. The hopping of the feet is induced indirectly by the motions of the upper body. Control of this novel mechanical structure has been investigated for walking, changing direction 65
Ö & Ü Ü Figure 5-9: Turning right, with a large turning rate, produced when Õ ×ß controller the robot can effectively turn in place.
Õ × Ø . With this
× Ö Ý&Ý Figure 5-10: Turning left, with a large turning rate, produced when Õ $ robot can effectively turn in place
Õ × Ø . Again the
and control of turning radius. Experimental evidence shows that although it lacks legs, surprisingly, such a robot is capable of efficiently producing numerous modes of locomotion, while maintaining dynamically stability. Future work will consider the use of more sensors for achieving further precision in control and dealing with increased uncertainty in real world applications.
66
Chapter 6 The BENDY Humanoid Robot Design and Control of Humanoid Robot Locomotion with Passive Legs and Upper Body Actuation1 In this paper we describe the design and construction of a humanoid robot which can walk with passive legs and only one actuated degree-of-freedom in the upper body. The humanoid robot BENDY, which has a 6 DOF lower body, 2 DOF torso, and springs connecting to shoulders and arms, is driven only by frontal plane oscillation of the torso. The mechanical design of the robot is such that the control of movement as well as balance is largely accomplished by the morphology. Thus, the control of the humanoid is simple, using only feed-forward control for rhythmic oscillation. Due to its minimum actuation requirements, in contrast to other humanoids, the BENDY robot is lightweight, energy efficient and robust.
6.1
Introduction
The field of humanoid robotics has grown rapidly in the past decades, producing several instances of advanced full body humanoid structures. Waseda University pioneered the field with the first development of the WABIAN (Waseda BIpedal HumANoid) [121]. Following this, several WABIAN robots were developed. WABIAN-RII [62], one of the most recent models has 43 DOF, and weighs 131.4 kg. The Technical University of Munich developed JOHNNIE [85] an anthropomorphic biped robot which has 17 DOF and weighs 37 kg. Honda developed some of the most sophisticated humanoid robots [45]. The HRP-2P [54] has 30 DOF and weighs 54.1 kg. The Honda P3 has 28 DOF and weighs 130 kg. Finally ASIMO [46], which is a downsized version of the P3 has 26 DOF, but weighs 43 kg. There are also more lightweight robots. The Sony SDR-3X [59] has 24 DOF and weighs 5 kg. Fujitsu has also developed a small lighweight humanoid robot, the HOAP-1 which has 20 DOF and weighs 6 kg. Although quite impressive, these robots are characterised by several actuated degrees of freedom requiring accurate control algorithms for walking. 1
Paul, C., Yokoi H., and K. Matsushita (2004) “Design and Control of Humanoid Robot Locomotion with Passive Legs and Upper Body Actuation”, International Symposium on Robotics Paris, France.
67
Figure 6-1: The BENDY robot The control algorithms used for such robots were originally based on the ZMP manipulation paradigm developed for biped walking by Vukobratovic [116]. The ZMP, or the zero moment point is a fictitious point in the (horizontal) ground plane, where the total ground reaction force would have to be applied in order to produce zero roll or pitch moment on the trunk. A biped robot can be stabilized by indirectly manipulating the ZMP to remain within the support area, by following appropriate joint angle trajectories. Thus, ZMP manipulation formed the basis of most trajectory generated biped controllers. Recently more real-time approaches have been developed. In these methods, horizontal trunk motions are often directly used to correct for rotational moments on the trunk and stabilize the system. However, there is category of biped robot which does not require precise balance control. The passive dynamic walker, originally developed by McGeer [69], is a robot in which the physical structure itself enables it to maintain balance and walk down a slope. The original design for such a robot had straight legs, curved feet and walked in 2D. Later it was shown that variations on this structure such as a biped with knees [70] or a compass like structure [32], could also acheive passive dynamic walking and that this could also be extended to 3D [15]. The success of these robots suggested that the principle of passive dynamic walking could also be used in active walking. This was successfully demonstrated in the lower body biped robots BAPS and MIKE developed at TU Delft. BAPS is a 3D lower body biped robot in which walking is acheived with passive knees and phasic actuation provided by McKibben pneumatic actuators [115]. The robot MIKE is a 2D walking robot, also with passive knees and ground contact dependent actuation using pneumatic actuators at the hips [120]. Although the concept of passive walking was applied to active lower body bipeds, it has not been extended to anthropomorphic humanoid robots. The BENDY robot, which
68
will be described in this paper, is one of the first robots2 to apply the principle of passive motion to a humanoid body plan. It is not simply a walker with passive knees and actuated hips such as MIKE, with an upper body attached on top. Instead, we show that using upper body actuation can eliminate the need for all actuation in the legs, in effect reducing the number of actuated degrees of freedom required for walking to one. Furthermore, we have shown that a simple feed-forward oscillatory control of the upper body, as previously used in our Stumpy robot [79], is sufficient to maintain balance and walk forward, eliminating the need for accurate balance control. The following sections will describe the design, construction and control of the BENDY robot. Experimental data will be provided to demonstrate the performance of the robot, followed by a discussion of its major advantages and limitations.
6.2
Robot Mechanical Design
The robot (Fig. 6-1) is approximately 1.9 kg in weight and 0.83 m in height. The upper body, measured from the waist up is 0.40 m in height, while the lower body is 0.43 m. The robot has 6 main DOFs in the lower body (Fig. 6-2). Each ankle, knee and hip has one DOF in the sagittal plane. Additionally, each hip has a slight possibility for passive movement in the frontal plane. The lower and upper legs are constructed as parallel link mechanisms (Fig. 6-3), which means that the knee and ankle joints are coupled, as well as the hip and knee joints. However, since the hip is also connected by a passive rotational joint to the waist shaft, the three degrees of freedom become decoupled from each other. Of these three DOFs, the hip joint is a passive rotational joint. The ankles and knees joints are constrained by springs (as shown in Fig. 6-3), which maintain the ankle joint at 90 Ó , and the knee joint at 180 Ó at equilibrium. The spring constants are such that a force slightly larger than that exerted by the weight of the upper body is required to compress the leg. The robot’s legs are connected to feet, which are large and flat metal plates. The upper body (Fig. 6-2) consists of a torso axis with an intermediate spring, connected to a shoulder axis with arms. The upper arms are attached to the shoulder axis by springs. The lower arm is connected to the upper arm via a passive rotational joint. The torso axis has two rotational degrees of freedom, enabling pitch and roll movements. The joint enabling pitch motion is passive and the joint enabling roll has the only actuator in the whole body. The upper body as described here, as well as the lower body, form the basic skeleton of the robot. The more important morphological characteristics which enable the robot to walk using only one actuator require more detailed explanation, which is provided in the following sections. 2
changed from the first in the original text. At the time of writing of the paper, the authors were unaware of the work of Paramonov and Lund [77], in which humanoid robots were also developed to walk using passive legs. Although these robots differed in morphology from BENDY to a large extent, they nonetheless used a similar concept.
69
R P
P
P
P
P
P
P
z
y x
Figure 6-2: Degrees of freedom of the robot. P indicates a passive pitch joint, and R, a passive roll joint. The zig-zag lines in the upper body indicate springs.
6.2.1
Trunk Balance Mechanism
In the traditional design and control of a humanoid robot, the upper body is balanced through control of the leg and trunk actuators. However, in the BENDY robot, the trunk is balanced through the morphology. As shown in Fig. 6-2 the trunk is connected to the waist via a passive rotational joint, in the sagittal plane. This means that unless constrained, the trunk either rotates forwards or backwards with respect to the legs, thereby causing the robot to fall. In order to prevent this, a fine balance has been struck between the rotational torques exerted on the upper body through the morphology. A closeup of the actual mechanical construction of the waist is shown in Fig. 6-4. The trunk is attached to a connector, which freely rotates around the waist shaft. Two smaller metal rods are also attached to this connector, such that the trunk, and the two rods roughly form an inverted Y configuration. The rod extending to the posterior of the robot has a counterweight with a mass of approximately 300g, creating a rotational torque on the trunk towards the posterior. The rod extending in the anterior direction has a hole in it through which a nylon cable is threaded. This cable wraps around and attaches to the right and left upper leg links, applying a torque on the upper body in the anterior direction. Thus, due to the opposing torques, the upper body can remain balanced during walking.
6.2.2
Cable Tendon Mechanism
When the upper body oscillates back and forth in the frontal plane, it initiates a rocking motion at the lower body such that the foot opposite to the direction of inclination is slightly lifted off the ground each time. But the level of ground clearance is not sufficient to initiate a swing. Thus, a cable tendon mechanism (Fig. 6-5) is used to directly amplify the effect 70
UPPER LEG
spring parallel link mechanism
LOWER LEG
nylon cable
FOOT
Figure 6-3: Leg construction TORSO RIGHT UPPER LEG connector
àâá àâá àâá âá âá âàâàà àâá àâá àâá àâàáâá àâá àâá â à à àá á à á à â â â âààâ
waist shaft
counterweight
tensioned nylon cable
LEFT UPPER LEG
Figure 6-4: Waist Construction of the upper body actuation on the feet. The mechanism consist of a pair of tensioned nylon cables which run between the actuated torso shaft, and an attachment point located on the outer edge of each of the foot plates. Thus, when the upper body is inclined to the left, for example, it pulls on the right leg to cause a certain amount of leg contraction as well as adduction in the frontal plane. This creates enough ground clearance to enable a swing. The attachment point of the cables on the feet were tuned through trial and error, to optimize the height of ground clearance.
6.2.3
Passive Swing
When a leg is on the ground, the morphology and the position of the CoG of the body determine that the waist is slightly anterior to the ankle, i.e. the leg is slightly slanted
71
TORSO
servomotor
LEFT FOOT PLATE UPPER LEG
cable attachment point
LOWER LEG nylon cable FOOT PLATE
Figure 6-5: Cable Tendon Mechanism forward. Thus, when the leg is freed from the ground it acts as a pendulum which is released from a non-zero initial angle and swings forward with respect to the waist, effectively causing a step.
6.2.4
Shoulder Mass Spring System
As shown in Fig. 6-2 the torso axis is not entirely rigid, but has an intermediate spring. This spring enables the shoulder and arms to act as a mass spring system in response to the movement of the torso. Although it is a coiled compression spring, being partially unconstrained lengthwise enables it to act as a torsional elastic component. Thus, lateral inclination of the torso to one side, causes the shoulder and arms to swing further relative to the torso axis, amplifying the effect of the inclination on the body. Furthermore, the stored energy from the sideways swing is released during the return swing, reducing the torque required by the motor for this motion. Using this mechanism, the motions initiated by the fairly weak servomotor can be amplified. However, the presence such of a mass spring system determines that this effect does not occur at all frequencies. At some frequencies the inclination of the shoulder is roughly in phase with the inclination of the torso, which serves to amplify the effect of the oscillation. At other frequencies, the shoulder may bend in a direction opposite to that of the inclination of the torso, in which case, the oscillation is diminished.
72
Figure 6-6: The H8 3664F microcontroller board
6.3
Control
The control of the robot is accomplished by an on-board Hitachi H8 3664F microcontroller board as shown in Fig. 6-6 The 3664F is a 16 bit microprocessor, operating at approximately 20MHz. This controller is used to alternately generate PWM signals for the servomotor corresponding to the desired maximum and minimum amplitude of oscillation. The switch between these two position commands takes place after a prespecified time interval corresponding to the desired frequency of oscillation. The motor at the torso is a HiTEC HS-5945MG high torque digital servomotor. The motor weighs 56 g and can exert a torque sufficient to rotate 13 kg at a maximum of 460 rpm. Although the change in the position command between the maximum and minimum positions is instantaneous, it takes the servomotor attached to the upper body a finite amount of time to switch between the two positions. However, usually the controller is operated in a range such that this time is usually smaller than half a period of oscillation, so that there is sufficient time to acheive the desired position while maintaining the desired frequency. Thus, it can be assumed that in the results presented below the desired frequency and amplitude correspond closely to the actual frequency and amplitude acheived.
6.4
Experimental Results
In order to demonstrate the range of performance of the robot, tests were performed at various amplitudes and frequencies of upper body oscillation. Three different periods of oscillation were selected within a range in which moderate performance had been observed: 1.2 s, 1.4 s and 1.8 s (corresponding to 0.71 Hz, 0.62 Hz, and 0.71 Hz respectively). Five different values of amplitude were selected based on similar criteria: ã Ò Ó , ãä åçævÓ , è6æGÓ , æ ½ å}æGÓ and é Ò Ó . For each combination of amplitude and frequency, the robot was allowed to walk for 20 secs, and its trajectory was recorded using an overhead camera which tracked an LED attached to the waist at 10 Hz. The best performance, acheived at an amplitude of 45 Ó and frequency of 0.71 Hz, was a speed of 0.037 m/s. The trajectory of the robot acheived in this condition, is shown in Fig. 6-7. (The video of the robot shows a comparable gait cycle.) In order to observe the temporal relationship between the upper body and the feet in this case, four force sensitive resistors (FSR) were attached under each foot plate. The data from these sensors was acquired at 100 Hz using a data acquisition card which was sychronized with the switching of the H8 controller. Fig. 6-8 shows the data collected from 73
90
80
70
60
50
40
30
20
10
0
−10
0
10
20
30
40
50
60
70
80
90
Figure 6-7: Trajectory of the robot walking with oscillation amplitude of è6æGÓ and frequency of 0.71 Hz. The trajectory has been obtained by tracking an LED attached at the robot’s waist with an overhead camera.
Table 6.1: Results: This table shows the distance travelled by the robot in 20 seconds, under varying conditions of amplitude and frequency of oscillation. Distances are shown in cm. Frequency Amplitude 0.56 Hz 0.62 Hz 0.71 Hz 30 Ó 37.5 Ó 45 Ó 52.5 Ó 60 Ó
9.29 8.73 30.20 29.80 30.38
68.95 45.55 41.01 51.46 47.74
59.20 43.38 73.57 57.45 63.22
the foot sensors with respect to the upper body actuation3 . In each of the other trials, the robot also walked approximately along a straight line and the difference between the initial and final position could be used to calculate the distance travelled. These distances are presented in Table 6.1. For visualization purposes, a plot of the speed with respect to frequency and amplitude is shown in Fig. 6-9. 3 The upper body position is equivalent to the position of the servomotor, which is indicated in the top graph of Fig. 6-8.
74
Figure 6-8: Data from foot contact sensors plotted with respect to upper body inclination. The top graph shows the upper body position with a maximum value representing inclination to the left and minimum value, inclination to the right. The middle graph plots the activation of the four foot contact sensors on the left foot, and the bottom graph plots the activation of the four foot contact sensors on the right foot.
6.5 6.5.1
Discussion Overall Performance
The results show that the robot can walk forward using only one actuator in the upper body, at various values of amplitude and frequency. The best performance acheived is a speed of 0.037 m/s, which although relatively slow, is sufficient as a proof of concept of walking with upper body actuation. Fig. 6-7 which shows a typical trajectory of the robot waist during walking, demonstrates how the upper body movement induces motion in the lower body. As can be seen, the trajectory of the waist zig-zags as it moves forward . This indicates that as the upper body oscillates, it causes the waist to swing from side to side. Since there is no passive roll degree of freedom between the waist and the legs, the swinging of the waist in the frontal plane almost directly translates to the rocking of the lower body, so that one foot is lifted slightly off the ground each time. The phase relationship between the upper body movement and the feet can be seen in Fig. 6-8. For the most part, when the upper body inclines to the left (motor at maximum amplitude value) the left foot contact sensors fire, while the right foot contact sensors become silent. This indicates that the weight of the body has been transferred to the left foot and the right foot has been lifted off the ground. The opposite is observed when the body inclines to the right. Thus, the rocking of the lower body is in phase with the upper body. This mode of walking was observed in most of the trials performed above. Once the foot has been lifted off the ground, it swings forward passively. The size of this motion is relatively small due to two factors. Firstly, the initial angle of release is small so the potential energy is not very high. Secondly, the cable tendon mechanism constrains the movement. Although this results in a small stride length and slow pace, it ensures that the foot is placed so that CoG of the body remains within its support area in the sagittal 75
0.04 0.035
speed (m/s)
0.03 0.025 0.02 0.015 0.01 0.005 0 0.75 60
0.7 55 0.65 frequency (Hz)
50 45 0.6
40 35 0.55
amplitude (Deg)
30
Figure 6-9: A plot of the average speed acheived by the robot under varying conditions of amplitude and frequency. plane. Thus, the gait is statically stable.
6.5.2
Relationship to frequency and amplitude
As can be seen in Table 6-1, there is no general monotonic relationship between amplitude of oscillation and walking speed. In the third column, for example, the speed alternately increases and decreases as the amplitude is increased. However, the amplitude of oscillation is not completely inconsequential. This can be seen from the first column of Table 1, where a simple change of amplitude from 37.5 Ó to 45 Ó can lead to a dramatic change in performance. The main reason for this change is that at a low amplitude and frequency of upper body inclination, one problem is that insufficient tension is created in the the cable tendons. Increasing the amplitude in this case, enables sufficient force to be exerted on the cable to lift the foot from the ground and take a step. If the only role of the upper body inclination was to create tension in the cable tendon mechanism, however, only the amplitude would have an effect on the performance: the further the feet were lifted off the ground, the longer would be the duration of the step. The fact that frequency can play a large role in the performance, as seen for example in the difference between conditions [30 Ó , 0.56 Hz] and [30 Ó , 0.62 Hz], bears evidence to the fact that the dynamics are also important for the movement of the lower body. There are two main factors which contribute to the dynamics: the inertial moments generated by the rotation of the lower torso, and torques generated by the torsional mass spring system of upper torso (shoulder and arms). The interaction of these factors give rise to a nonlinear dependency of speed on frequency as well, as seen in the results.
6.5.3
Improvements
Stride Length, Walking Speed Currently, the maximum stride length is 0.025 m (step length 0.05 m) and maximum speed 76
is 0.037 m/s. Although this is sufficient as a proof of concept of walking with upper body actuation, it is insufficient from a practical perspective. Future work will focus on optimizing the mechanical design to improve this characteristic. Increased Passive Movement The forward step of the robot is generated passively to the extent that leg swings forward as a pendulum due to being released from a non-zero initial angle. However, it is constrained from performing a full passive swing by the cable tendon mechanism. In the next design, it will be attempted to enable the legs to swing more passively. Upper Body Movement Currently, the upper body movements are much more exaggerated than that of anthropomorphic gait, mainly due to the fact that the upper body is light. In a next version, a heavier upper body will be designed, so that a smaller motion of the upper body produces a larger inertial effect on the lower body. This would make the morphology as well as the gait more anthropomorphic.
6.5.4
Role of the upper body
In the traditional approach to the control of biped or humanoid locomotion, the problem is conceptualized as the control of an inverted pendulum. In this approach, the main responsibility for the control of balance and movement lies in the joints of the lower body, such as the hips, knees and ankles which must therefore be actuated. The upper body is merely thought of as a mass to be carried along. In our work, however we invert this fundamental paradigm of biped control, and consider the upper body as the main seat of balance control and movement. Thus we use the inclination of the upper body in the frontal plane to bring the center of mass of the body above the supporting foot, and free the other foot from the ground to move forward. This simple mechanism eliminates the need for balance control as well as actuation in the lower body.
6.6
Conclusion
This paper described the design and construction of the BENDY humanoid robot, which has a passive lower body, and walks using oscillation of its torso in the frontal plane. The paper shows4 that a humanoid robot can be controlled to walk on level ground with passive legs and only upper body actuation. Furthermore, it shows that only one actuated degreeof-freedom is sufficient to acheive such walking. The robot also illuminates a new approach to balance control for both biped and humanoid robots. Instead of conceptualizing the upper body as a mass to be balanced by the motions of the lower body, the upper body is used to drive the motion of the lower body such that its resultant motions help to maintain balance. The results show that as long as 4
changed from shows for the first time in the original text, in light of [77] which was recently brought to the author’s attention.
77
the lower body motions induced remain within a certain region of stability, balance can be maintained at various walking speeds.
78
Part III Spinal Control
79
Chapter 7 Neuro-Musculo-Skeletal Model Development of a Human Neuro-Musculo-Skeletal Model for Investigation of Spinal Cord Injury1 This paper describes a neuro-musculo-skeletal model of the human lower body which has been developed with the aim of studying the effects of spinal cord injury on locomotor abilities. As compared to other neuro-musculo-skeletal models which have only ensured that the overall system behavior resembles locomotion, our model ensures that the behavior is biologically relevant both at the system and at the subcomponent level, by independently validating the functionality of each of the subcomponents with respect to their biological counterparts. The model is thus suitable for the in-depth investigation of clinical questions related to spinal cord injury. Preliminary experiments are performed to address the questions of a) the significance of spinal reflex modalities for walking and b) the relative criticality of reflex modalities. The results of these experiments shed new light on the role of the known reflex modalities in human walking. In addition, they show that the model can be used to provide insights into the role of neurophysiological pathways in the regulation of locomotor parameters (such as speed and stepping frequency). Such a model will also have applications in clinical diagnosis, as it can be used to identify the internal state of the system which provides the closest behavioral fit to a patient’s pathological condition.
7.1
Introduction
Spinal cord injury can be a debilitating event, often leading to severe loss of locomotor functionality. In some cases, this functionality can be recovered with appropriate clinical care (Rossignol 2000). It has been shown that a form of rehabilitation therapy called treadmill training can improve such recovery (Dietz 94). However, so far in clinical practice, a uniform training regime has been applied to all patients irrespective of the underlying neural cause of their pathology. As not all injuries lead to the same neural lesions, this 1
Paul, C., Bellotti, M., Jerzernik, S. and A. Curt (in prep) “Development of a Human Neuro-MusculoSkeletal Model for Investigation of Spinal Cord Injury”, for Biological Cybernetics
80
is not optimal. The overall recovery of spinal cord injured patients could be greatly improved by identifying the underlying neural cause of a particular pathological condition, and customizing the therapy regime to fit the specific lesion. However, in order to do this, it is first necessary to gain a deeper understanding of the neural mechanisms of the spinal cord involved in locomotion, and the ways in which injury to these mechanisms leads to pathological conditions. During the last three decades, advances were made in understanding the neural mechanisms involved in the control of locomotion, both with respect to rhythm generation (Delcomyn 1980; Grillner 1985) and adaptive reflexes (Pearson 1995; Prochazka 2002) in the spinal cord. However, these advances have not yet been integrated into a sufficiently accurate model representing the mechanisms of the spinal locomotor circuits and their function. Such a neuro-musculo-skeletal model would be a necessary tool to answer questions about neural lesions in the spinal cord, and their effects on locomotor patterns. Several partial spinal cord models were developed. Some represented mainly the behavior of central pattern generators (Prentice 1998). Others investigated the emergence of locomotion through the interaction of central pattern generators with the musculo-skeletal system (Taga 1991; Taga 1995). In others, more focus was placed on the representation of the reflex pathways and their stabilizing effects on the musculo-skeletal system (Chou 97). However, these models were not complete. Recently, development of more complete models of the spinal cord integrated with the musculo-skeletal system were attempted. In the model by Wadden and Ekeberg, reflex pathways were combined with neural phase generators which regulate the reflex gain modulation in each of four phases of the gait cycle (Wadden and Ekeberg 1998). In another model by Rybak, the half center model of a joint CPG was combined with local Ia, Ib and cutaneous afferent pathways, to produce gait in an accurate musculo-skeletal model of a quadruped hindlimb (Rybak 2002). In a model by Ogihara, central pattern generators were integrated with Ia, Ib and cutaneous pathways, which had interconnections between them whose weights were determined with a genetic algorithm (Ogihara et al 2001). Although these models generated overall behavior resembling locomotor patterns, it was not ascertained whether the models were topologically or functionally accurate at the level of the subcomponents. Thus they could not be used to address numerous issues which are of clinical relevance. Firstly, since the subcomponents of the system may not have biologically accurate functions, removal of these components from the system may have lead to behavior which were simply artifacts of the model, and did not accurately represent the result of removal of these components from the biological system. Thus, they could not be used to study the significance of these neurophysiological pathways. Secondly, they could not be used to address questions of relative criticality of the reflex modalities, or in other words which modalities must be more strongly activated than others. Since the functional contributions of the reflexes in these models may have been different from the real system, they may have wrongly predicted how strongly a particular reflex modality was required to be active. Thirdly, the inaccuracy in functional contribution may have lead to system components being used for alternative purposes than in the real system, providing misleading conclusions about their emergent effects on global behavior. Finally, such models were also not suitable for lesion studies in which parts of the network are eliminated in an attempt to match the resultant behavior with a pathological condition for 81
Cerebral input NSCM
êa -m ë otoneurons
CPG
í Proprioception
+
BHBM
Reflexes
Excitation-Contraction Coupling
NSCM
Sensory Receptors
Muscle Activation
î Golgi tendon ï organs
Muscle Dynamics
ì Muscle Forces Muscle fibers length/velocity
ð
Muscle Cutaneous spindles receptors
BHBM
Body Dynamics
Kinematics
Kinematics Reactions
Environment
Figure 7-1: Diagram indicating the overall structure of the neuro-musculo-skeletal model. clinical diagnosis. A neuro-musculo-skeletal model was required in which not only the global behavior was accurate, but the representation of the subcomponents was also biologically relevant. In the development of such a model two conceptual stages of validation were necessary: one at the subcomponent level, and the other at the global behavioral level. This methodology was adopted for the construction of the model. The main components of the neural model are the central pattern generator and the Ia, Ib and cutaneous afferent pathways, modelled according to the known topology of the pathways from neurophysiology. The neural model interacts with the musculo-skeletal model, which is closely modelled according to the known biomechanical properties of the human body. The interaction of each of the neural subcomponents with the musculo-skeletal system, has been independently validated. These components have then been integrated to produce the overall model which was validated and displayed stable walking behavior. The model was then used to perform experiments to investigate two previously unanswered questions on the role of reflex pathways in walking. The first set of experiments was conducted to study the significance of each of the reflex modalities, or in other words determine whether each reflex modality is necessary for walking. The second set of experiments focused on studying the relative criticality of the reflex modalities, in an attempt to determine which modalities required strong activation during walking and which required only weak activation. The results of these experiments shed new light on these previously unexplored questions and additionally demonstrated the potential of such a model for illu82
minating the emergent effects of local neural pathways on global system behavior. The next section, Section 7.2 will outline the topology of the neural spinal cord model. Section 7.3, will then describe the construction of the biomechanical musculo-skeletal model. Section 7.4 will focus on the first step of validation, that is validation of the individual neural subcomponents. Section 7.5 will describe the results of the validation test at the global behavioral level. Section 7.6 will describe the experiments on significance and relative criticality of reflex pathways and present the results. Section 7.7 will discuss the implications of the results and describe the scope for future work, and Section 7.8 will summarize with conclusions.
7.2
The neuro-musculo-skeletal model
The overall-structure of the neuro-musculo-skeletal model is shown in Figure 7-1. The model is comprised of two components, the neural spinal cord model, and the musculoskeletal model. The interaction between these two components is shown by arrows. The following sections will independently describe the neural spinal cord model and the musculoskeletal model.
7.2.1
Neural Spinal Cord Model
There is a strong evidence suggesting the existence of a spinal central pattern generator in vertebrates which is responsible for rhythmic muscle activation patterns even in the absence of afferent inputs (Delcomyn 1980; Grillner 1985). Due to evidence suggesting that bipedal walking uses mechanisms involved in quadrupedal locomotion (Dietz 2002) and measurements of rhythmic movements in spinal cord injured patients, it is also believed that a Central Pattern Generator may exist in humans (Calancie 94). In our neural spinal cord model, a Central Pattern Generator module was implemented for each joint, based on the mutually inhibitory half-center model proposed by Brown (Brown 1914). Spinal reflexes are known to play an important role in the regulation of locomotion (Pearson 1995). They are involved in shaping the inputs of the centrally generated rhythm (Rossignol et al 1988; Pearson 1997) and are also modulated during locomotion (Stein 1988). Currently the spinal reflexes have been implemented in our model according to the known topology of these pathways as described in (Kandel 1991). The next sections will describe the different components of the neural spinal cord model in detail. Neurons and Sensory Receptors The neural spinal cord model was designed as an artificial neural network with recurrent connections. Each neuron received inputs weighted by synaptic weights, and produced an output value according to a piecewise-linear activation function. The equations of the neuron model are given by (1) and (2):
83
Õ%ñ ¯ ó ò ñ ô ¢ ô ³ «eñ ô\õ Gö ù
º Ï Õá¾ ¯
÷ø øú Õ
¤ Ò ¤ »
¤
(7.1)
Õû Ò Ò Ý Õ Ý » Õü'»
(7.2)
where ÕÄñ is the internal activation of each neuron, ¢ ô are the inputs, ñ ô the connection ö weights, «eñ the is threshold or bias and Ï the neuron output. This neuron model was used to implement two kinds of neurons: spinal interneurons and alpha motoneurons. The interneurons usually recieved inputs from afferent neurons or other interneurons and conveyed their output also to other interneurons or alpha motoneurons. The interneurons connected to form various neural modules such as central pattern generators and reflex pathways which finally activated the alpha motoneurons. The alpha motoneurons then activated their corresponding muscle groups. The neurons which conveyed afferent inputs however, were modelled differently. The muscle spindle receptors had a static and a dynamic component, which correspond to muscle length and muscle velocity respectively. The static and dynamic gamma motor neurons were not modeled and it was assumed that the response of the primary and secondary endings of the spindle afferents have a constant ratio with the respect to the dynamic and static components. Also, it was assumed that the appropriate alpha-gamma coactivation maintains the sensitivity of these receptors at a constant value. The Golgi tendon organ activation was also formed by the contribution of static and dynamic fibers which respond to muscle force and the rate of change of muscle force respectively. The coefficient of the static fiber was set relatively high compared to the dynamic coefficient. The contribution of these two fibers were summed to produce the total Golgi tendon organ activation. The cutaneous receptors output a signal which also had two components: one proportional to the ground reaction force and the second proportional to the derivative of the ground reaction force. The general form of receptor activation used can thus be described by the following equations:
ý6þÿ ñ º ÿ ¾ 7 ¯ Û
¤
º º ÿ ¾ ³
Û
¯
Û ¯
º Û ¤
¤ º ÿ ¾ ü Ò ¤ º ÿ¾ Ý Ò
¼/ý ³
¼/ý ¼/þ
"£K¾À¼
¤
¤
º º
º ÿ¾ ÿ ¾ü Ò º ÿ¾ ÿ ¾ Ý Ò
º ÿ¾ ÿ ¾
(7.3)
(7.4)
(7.5)
where þ is zero for muscle spindle receptors and ý , and þ are positive constants in the º other cases. The ÿ¾ is equal to the normalized muscle length, the muscle tendon force, 84
Tonic Input Delay
Delay Half−center
Half−center
HC
HC
B E
Extensor Motoneuron
Flexor Motoneuron
F S1
CPG Hip Right
CPG Knee Left
CPG Knee Right
CPG Ankle Left
CPG Ankle Right
S2
A Extensor Muscle Group
CPG Hip Left Delay
Flexor Muscle Group Delay Circuit Diagram
Figure 7-2: (a) Central Pattern Generator circuit for a single joint (A represents an excitatory connection and represents an inhibitory connection) (b) Phase relationships between hip, knee and ankle oscillators (A represents an excitatory connection and represents an inhibitory connection) and the ground reaction force for the muscle spindle, Golgi tendon, and cutaneous receptor pathways respectively. For all three kinds of receptors, the activation is normalized such that the receptor output is a signal which is between 0 and 1. If the lengths or forces become extreme, the sensor activations saturate at a value of 1. Central Pattern Generator The Central Pattern Generator network consists of six neural oscillator units, one for each hip, knee and ankle joint. Figure 7-5(a) shows a single oscillator unit. HC Flex and HC Ext are mutually inhibiting half-centers. They are connected to delay circuits, which determine the amount of time each half-center remains active. For the hip and ankle, the extensor and flexor side delay circuits are identical such that each is active for 50% of the cycle. For the knee, the extensor is activated for 67% of the cycle and the flexor for 33% of the cycle. The frequency of oscillation of the neural centers is set to 1 Hz.On the top, there are two input areas which are responsible for initiating the oscillatory dynamics. At the bottom are the motor neurons which receive their activation from the oscillator. The tonic input of the flexor half-center on the left side is the same as that of the extensor half-center on the right side, and vice versa. This ensures, that at the beginning of walking, the legs begin to move in opposite directions, as is necessary for the generation of a walking pattern. The joint specific input areas are also responsible for implementing time delays between the different joints, since oscillations of the hip, knee and foot are not synchronous, but phase shifted in time. The six oscillator units are not directly connected with each other, but the timing relationships are maintained by the common tonic input; the hip oscillator is activated first, then the foot with a delay of about 1/10 of a step cycle, and finally the knee with a delay of about 1/5 of a step cycle. 85
Golgi Tendon Organ
Right Foot Cutaneous Receptor
Stretch Receptor
LEFT Extensor Muscle Group
E Extensor Motoneuron
Extensor Muscle Group
RIGHT E
F
Flexor Muscle Group
Renshaw Cell
Rn
Ia
Ib
Rn Renshaw Cell
Ia Interneurons
Rn
Ib
Ib
Rn
Rn
Ib
Ib
Rn
Ib Interneurons
Rn
Ia
Ib
Rn Renshaw Cell
Renshaw Cell
Flexor Muscle Group
Flexor Motoneuron
F
Golgi Tendon Organ
Flexor Muscle Group
Extensor Muscle Group
F
Stretch Receptor
E
Left Foot Cutaneous Receptor
Figure 7-3: (a) Spinal reflex pathways corresponding to muscle spindle and Golgi tendon organ receptors ( represents an excitatory connection and represents an inhibitory connection) (b) Reflex pathways corresponding to right and left cutaneous sensors ( represents an excitatory connection and represents an inhibitory connection Muscle Spindle Pathways As described in Section 7.2.1, the muscle spindle receptors have a dynamic and a static component. This is modelled according to the known functional anatomy of the muscle spindle afferents. It has been confirmed that there are two kinds of sensory endings in the muscle spindles. The primary ending consists the group Ia axons, which are mainly responsible for conveying dynamic information, the rate of change of muscle length. The secondary ending consists of the group II axons which are mainly responsible for conveying static information on muscle length. Both these afferents make monosynaptic excitatory connections to motoneurons innervating the homonymous and synergist muscles of a joint, mediating a stretch reflex (Kandel 1985). Thus, the stretch reflex has both tonic (static) and phasic (dynamic) components. However, the effect of the tonic component is usually significantly lower than the phasic component. Thus, in our model, the stretch reflex has been modelled only as a monosynaptic excitatory pathway from the primary spindle afferent to the alpha motoneuron (Fig. 7-3a). The spindle receptors are also responsible for reciprocal innervation, through a spinal interneuron. This reflex is modelled by two pathways: 1) a disynaptic pathway from the group Ia afferents to the alpha motoneuron via an inhibitory interneuron 2) a polysynaptic inhibitory pathway from the group II afferents. (Fig. 7-3a). Golgi Tendon Organ Pathways It was originally thought that the Golgi tendon organ receptors, are connected via a Ib inhibitory interneuron to provide a negative feedback mechanism for regulating muscle stiffness (Kandel 1985). For large values of muscle tension, and in some tasks, this is observed to be the case. However, in locomotion a reflex reversal is observed in which the Ib afferents are used in positive feedback loops (Prochazka 1997; Prochazka 1997). In our network, Ib afferent connections were modelled as two pathways which travel over separate Ib
86
interneurons (Fig. 7-3a). The threshold of the first Ib interneuron is very high, corresponding to extreme values of muscle tension. This interneuron is connected via an inhibitory connection to the ipsilateral motoneuron of the same muscle. The threshold of the second Ib interneuron is 0, and is connected to the motoneuron via an excitatory connection. Thus, for all low values of muscle tension this pathway is activated. This pathway inhibits itself when the value of muscle tension reaches the threshold of the inhibitory pathway.
Table 7.1: Anthropomorphic Parameters of the Musculo-skeletal model have been set according to Winter, 1990. Part Mass (kg) Length (m) Moment of Inertia WHOLE BODY 80 1.80 "! #%$ & ')(*&*+,#.-/ "!1032 HAT 46 0.85 PELVIS 8 0.09 547698*:9;3<%#=$&>'@?#A-B476C8*:7;<"0 2 THIGH 8 0.35 D!> ;3E7 F#%$ & '@+HGI+,#.-J!> ;3E7 A0 2 SHANK 3.7 0.44 5!W#%$ & 'PO"X*(Y#.-BU9V9V>!Z032
Cutaneous Pathways Three types of cutaneous responses have been observed. In the plantar reflex, stroking the plantar surface of the foot with a sharp surface from heel to toe leads to a plantar flexion of the toes. In the extensor thrust reflex, light pressure on the plantar surface of the foot leads to excitation of the extensor muscles of the same leg. Finally, a painful stimulus to the surface of the foot leads to the flexion withdrawal reflex which causes the contraction of the all the muscles of the same leg (Kandel 1991). In our model, the foot was modelled a rigid body so the plantar reflex was not relevant. Furthermore, as the aim was to model walking under normal conditions, and not under unforeseen conditions involving painful cutaneous stimuli, the flexion withdrawal reflex was also not relevant. Thus, only the extensor thrust reflex pathways are modelled. The extensor thrust reflex is modelled as two disynaptic excitatory pathways from the foot cutaneous receptors via Ib interneurons (Fig. 7-3b). The right cutaneous receptor is connected to a Ib interneuron pathway which excites the extensor muscles of the right leg. It is also connected to a second Ib interneuron which lightly excites the flexor muscles of the left leg.
7.2.2
Musculo-skeletal Model
The Biomechanical Human Body Model (BHBM) was developed for the use in combination with the Neural Spinal Cord Model described above. The BHBM is bidirectionally coupled to the NSCM to simulate complex sensory-motor interactions (referring to Fig. 7-1 the BHBM comprises: excitation-contraction coupling, muscle dynamics, body dynamics, and model of environment). A feedforward dynamic simulation of the BHBM, which is 87
fed back to the NSCM, yields the kinematics and kinetics of human locomotion for a given [ -motoneuron activation. Skeletal Model: The skeletal model consists of seven segments: the feet, the shanks, the thighs and one segment for the head, arms, and trunk (HAT). The BHBM was implemented in MATLAB using m-scripts and m-functions, and broadly utilizes the MATLAB Symbolic Toolbox. The variables that allow the description of the skeletal system are angles and rotational velocities of the segments, and the displacements and velocities of the upper extremity of the HAT: thus the overall body dynamics has an order of 18 and can be written as: \
$^]_0K`1c a] bedf$] gK]*h 0Bikjml npo q^rtsbujvswvx \ the state vector, $]*0 is a symmetric, positive definite matrix
(7.6)
of inertia, df$]*0 where ] is is a vector representing the Coriolis, centrifugal and gravitational terms, jplKnpo q^rPs represents the torques generated by the muscles, and jps wpx is a term modelling the interaction with the environment (e.g. the ground reaction forces, the external body unloading system). The positive sign of model variables (joint torques and angles) was defined as shown in Fig. 74. The anthropometric parameters used in the musculo-skeletal model consist of estimates of the geometrical dimensions of different segments and the corresponding kinetic data. These parameters were taken from (Winter 1990) and are summarized in Table 7.1.
Figure 7-4: Definition of the variables of the skeletal model. The jml npoqrts term is calculated by using the muscle model (see section 7.2.2) driven by the motoneuron activations that are input from the NSCM. The jps wpx term is calculated in the foot-ground reaction model and by considering the Lokomat treadmill conditions (see sections 7.2.3 and 7.2.4). The body kinematics ] is then obtained by integration of equation 7.6. However, model relationship given by eq.7.6 also contained an additional knee constraint model. This model provides an angular limit for the knee motion, which prevents the 88
knees from extending beyond the limit ]mlJy{z . The knee constraint model was implemented according to a standard nonlinear constraint model as described by the following equation: jvNJwSss|i
}H~i
}H~`S K$]SNJwvss u]v~Jb
]v~m0 L w_IJ }H~T$]vNJwSs sJu]v~ b
]p~m0 wM~ L wHTJ Fl y3zS NJwvss $ $
]v~m0 K$
]v~m00 wSM~
(7.7)
When the joint angle ]SNJwvss increases over the limit ]p~_
]p~ the moment jSNJwvss increases monotonically as an exponential deprived of the first N+1 terms of the Taylor series. The constant }H~ is calculated based on the value of the moment DlJy{zS NJwSs s when the angle ]SNJwvss reaches its maximum value ]v~ . The values of the constants were set to: ]p~ iQR*& deg,
]p~.iQ& deg, FlJy{zS NJwSs s/iQS&_& Nm, iG . Muscle model: For every joint, a single muscle representing the whole extensor muscle group and another one representing the whole flexor muscle group were implemented in the model. Since the skeletal model has six joints, and each joint has an extensor/flexor muscle groups, the whole model comprises twelve muscles. Each muscle was represented by a global muscle model consisting of two parts: the active contractile element and the passive visco-elastic element which act in parallel (McMahon 1984). The active contractile element was modelled by the equation of the standard Hill muscle model (Hill 1938). This equation describes muscle dynamics during voluntary > contraction. In the Hill-type models, the active muscle force l is a product of the muscle M6 activation l , and the values dMr and d>x which depend on muscle length l and muscle velocity x respectively. l
>
ik l
M6
`dMr$l 0J`Td>x_$^¡TlA0
(7.8)
The functions dMr and dMx implement the force-length and force-velocity relationships experimentally observed in human muscles. They were modeled as follows:
dMr $ lA0|i
>K$¢ `M£
«
¤luul/ ¥§¦©¨ £ l/ ¥§¦©¨Z`ª £
£ 0 £
(7.9)
£
£ £ x¬Z £ ® ¬>¯°p±"x¬ £ g T ¡ ¶ l µ·& x ¬C® ¬>¯°²9³´ x ¬ d>x_$^¡TlA0|i ¸s qqZb
x¬C® ¬>¯° ² x¬ ¹ ´ ³´ x ¬ ±"x ¬C® ¬>¯° g¡l·º»&
(7.10)
The length ¼¥¼¦¨ denotes the optimum length of the contractile element, that corresponds to maximum production of force; the parameter ª describes the width of the bell-shaped
89
dMr
curve and in our model was set to 1; the value of the constant ¢ is $&>'@&_(_0 to fulfill the equation: d7r $¼¥§¦©¨C` $3Q½eª%00¾i¿&>'P&H( When a muscle contracts (¡lÀµÁ& ), d>x follows the equation described in (Hill 1938). The parameter ¡l l y3z , which has a negative value, specifies the maximum shortening velocity at which the contracting force generation vanishes. à is a shape factor. When a muscle extends (¡lĺ& ), d>x follows the equation described in (Aubert 1956). The steepest slope occurs in the neighborhood of ¡lÅiÆ& and saturates at a high eccentric velocUǼÈÈ ity (which approximately equals sq^q ), which is the normalized amount of force ( U ¬>¯° ) reached at the lengthening velocity ¡liÉ¡l y3z . M6 The muscle activation l is calculated from the motoneuron activation fed to the excitation-contraction coupling model. The relationship between the model input-output is given by Eq. 7.11 . This equation describes the inverse Laplace transform of alpha motonueron activation filtered by a first order filter with gain Ê , zero Ë and pole Ì . This linear filter approximates the behavior described by the recruitment and excitation model of (Hatze 2001) and (Zajac 1989). It actually models the extent of depolarization of muscle cells due to neural stimulation: the stronger the stimulation the larger the region of recruitment. On the other hand, the electrical transmission from the alpha-motoneuron to the muscle fibre is a process that requires a finite amount of time. This was modelled through the effect of the pole Ì (which corresponds to a time constant Q Ì ). l
M6
$^ÍÎÏ%LBÐ@¨¤Ñ 0/i¿-
±7Ò Ó
Ê¿`
${Q.Ô ${QYuÔ
ËH0 ÌC0
`SÍÎÏ%L$Ô0
(7.11)
The rest lengths of all the muscles were implemented to be identical. The muscles had a varying maximum force capacities depending on the muscle group they represented. The maximal muscle forces used are listed in Table 7.2.
Table 7.2: Maximal Muscle Forces Muscle Max Force (N) Tibialis Anterior 1000 M. Soleus 500 Vastus Medialis 1000 Biceps Femoris 500 Tensor Fasciae Latae 500 Gluteus Maximus 1000
The second component of the muscle model, the passive visco-elastic element acts in parallel to the active contractile element. This component was implemented according to the model of Riener and Edrich (Riener 1999) which was experimentally validated on 47 human subjects. The passive component lÂ Õ for each muscle is given by the following equation:
90
47 lÂ Õ i
`T×7Õ¼±7Ò1b¢ « `T×9Õ b¢©Ø`T×9Õ ² Ò0 2 m Ö$ ¢ÙJb¢mÚÂ`T×9Õ±7Òb¢pÛB`T×7Õ b¢Ü`T×7Õ ² Ò0 Ià ¢mÝJbÞjMNJ ß wSs s 0 l $>Ö$ ¢SÒ1b¢
b
(7.12)
where the subscript á denotes the ankle, the knee or the hip index, and ×7Õ¼±7Ò , ×7Õ and ×7Õ ² Ò à are the relative angles of the corresponding joints. l is the moment arm of each muscle according to the muscle insertion point on the bone. The values of the constants ¢â were identified by Reiner and Edrich through experiments on healthy subjects. These values are listed in Table 7.3.The component j J ßN wvss was added to the equation of the knee only, and Ð ~± ~ ÒÙ~ ´ åpæç Ç§Ç Ñ ä equals ã 2ä 22 . This term was added since it allowed a better identification fitting (Riener 1999). 47 47 à For each muscle, the passive torque component j l is given by lÂ Õ ` l and the active à > M M component j l is given by the product l is given by the eq. 7.8. ` l where l The global torque jpl in eq. 7.6 was thus a summation of the active and passive torque components: jmliÉ l
M `
à
47 à lèb l/ Õ ` l
(7.13)
Table 7.3: Values of the constants of the passive muscular model part as identified for a healthy subject (Riener 1999) Constant Ankle Knee Hip ¢Ò 2.1016 1.800 1.4655 ¢ 0.0460 0.0034 2 ¢ « 0.0843 0.0352 0.0750 ¢Ø 0.0176 0.0217 ¢mÙ -7.9763 -3.971 1.3403 ¢mÚ -0.0004 -0.0226 ¢pÛ 0.1949 0.0495 0.0305 ¢mÜ 0.0008 -0.0128 -1.792 -4.820 8.072 ¢mÝ
7.2.3
Foot-ground reaction model:
As reviewed by G¨unther in his thesis (G¨unther 1997), a linear spring-damper constraint is not accurate enough to describe the dynamics of the foot-ground reaction . Therefore, we decided to adopt a non-linear foot-ground reaction model, which G¨unther showed to provide best fit to the experimental data. From a sensitivity analysis of a walking mechanical system, it has been shown that differences in the foot-ground constraint models are the most important causes of differences 91
between simulations and measurements. This is the case because of the impulsive nature of the ground reaction forces that may reach peaks of up to 2000N within few milliseconds. Thus, we have carefully chosen a foot-ground reaction model for our overall system that provides a good-enough fit for the reaction force data measured during foot-ground contact. Both, the spring and the damping components play an important role to define the actual shape of the reaction force. The best fit that we have adopted from [G¨unther 1997] was found to have the following form: KzS Õéi
Kñ Õéi
6 ǧ° ï° ,êczY`*ëíì7ÕC`>îtëíì7Õî EMǧ° ï° ,Ê=zY`¡zðZ`Mîtëíì7Õî 6 ǧ° ïô ,êcñ.`*ëíòIÕ7`MîtëóòIÕ3î EMǧ° ïô ,Ê=ñ.`¡ñ{ðC`MîPëóòTÕî
ëíì7Õéi
ìCõ©¥¥¨Zì7ö{÷¥nwSø
ëíòIÕéi
ò_õ©¥¥ ¨ZuòTö{÷¥nwSø
¡zðùi
¡zmúû^ûüCu¡T¨ý÷s yølKÕýrtr
¡ñ{ðùi
¡ñ3úû^ûü
(7.14)
where á is the foot index (left, right), KzS Õ and Kñ Õ are the ground reaction force components, ëíì7Õ and ëóòTÕ are the horizontal and vertical indentations, and ¡zð and ¡Tñ{ð the relative foot velocities with the respect to the treadmill motion. The equations for KzS Õ and Iþ Ú Kñ Õ represent nonlinear spring and damping reactions with ê.z%ikO¸`HQ& 2 , ê.ñYiÿQ_'tO=` « « Iþ Iþ Iþ Ø Ù Ò Ù Ú Ò Q& 2 ä , êYsz{¦©z iQ , êcs z{¦©ñciG ')( , Ê%ziÉOM`Q& Ô ä , Ê=s z{¦©zi , Ê=ñciG 'PR>`¼Q& 2 Ô G , and Ê=s z{¦©ñcik+>')( . The vertical indentation ëíòIÕ is negative if the foot is below the ground level. In the case of no foot-ground contact the force components Zâ3 Õ are zero. The variable ìCõ©¥¥¨ is the local variable with the respect to each initiation of the foot-ground contact (foot-ground variable ì7ö{÷¥ nmwvø ). This means that the variable ì7ö{÷¥ nmwvø is set anew at each new foot-ground contact. òTö3÷¥nwSø equaled zero.
7.2.4
Lokomat Treadmill Conditions:
At the University Hospital Balgrist the driven gait orthosis Lokomat is used extensively for rehabilitation of spinal cord injured patients (also in the context of this project) [Colombo et al. 2001; Jezernik et al. 2003]. The Lokomat system consists of a robotic orthosis with two legs, a harness, and a treadmill. The system is used for rehabilitation by producing walking by actively moving the subjects’ legs on a treadmill. The system also allows measurements of gait trajectories, forces, joint torques, and ground reaction forces on healthy and impaired
92
subjects. Subjects who have spinal cord lesions and cannot control their leg movements can benefit from this training because the spinal centers below the lesion can be stimulated by induced afferent activity. In the first sessions of the treadmill training, the patients are completely passive and the Lokomat completely controls their leg movements. After a few sessions, some voluntary muscle activation can be recovered. During Lokomat therapy, the subject is not required to control upper body posture since the hip is constrained and a harness supports the upper body. In order to easily compare the simulation data of the NSCM-BHBM model with clinical measurements made in the Lokomat system in healthy and impaired subjects, the Lokomat treadmill conditions were implemented. In our model, the body unloading was achieved by adding a vertical force to the center of mass of the trunk, the magnitude of which produced an approximately 60% body unloading. This also generated a torque that helped the upper body remain upright, eliminating the need for postural control. The treadmill condition was implemented as a constant horizontal shift at the foot-ground contact, that is, in eq. 7.15, the velocity ¡T¨ý÷sy3ø{lKÕýrtr was not zero, but had a positive constant value of 0.35m/s, which resulted in the modification of the term ì7ö{÷¥nwSø , which now linearly increased with time during the duration of each step.
7.3 7.3.1
Subcomponent Validation CPG Tests
In the CPG test, the central pattern generator was validated without the sensory pathways. In this test as an initial condition, the body was vertically suspended, and all the joints are at rest, and free to move. The behavior observed was that the legs start oscillating and after one cycle of motoneuron activation (around 1 sec) the movement converges to stable oscillation following a kinematic gait pattern.
7.3.2
Muscle Spindle Tests
In this test, as an initial conditions, the body was suspended and the left thigh was flexed to 90 , All the joints were fixed, except for the left knee. A perturbation was applied such that the knee extensor was stretched for 100 msec till it increased by 20% of its length. After the stretching, the muscle fiber length was set back to the value due to the geometrical constraint. The muscle spindle receptor is not silent before the perturbation, due to the static receptor activity. When the perturbation is applied, a peak is observed in the first part of the perturbation, due to the activation of the dynamic receptor, and it goes rapidly to a steady value, which is higher than the base firing rate (BFR). When the fibers contract after the external stimulation, the sensory activity goes back to BFR. A saturation of MN activation, due to the rapid change of the velocity of the muscle fiber, occurs in the very first part of the perturbation; after it, the MN activity is almost silent. The sharp motorneuron activation leads to a strong torque on the knee joint, corresponding to a sharp contraction of the extensor muscle. This leads to the observed increase in the joint angle, i.e. extension of 93
MN L Hip
MN R Hip
0.8
0.8
0.6
0.6
q L Hip
q R Hip
5
4.5
4.5 4 4 0.4
0.4
3.5
3.5
3 0.2
0.2
3 2.5
0
0
1000
2000
3000
4000
0
0
MN L Knee
1000
2000
3000
2
4000
0
1000
MN R Knee
0.4
2000
3000
4000
2.5
0
1000
q L Knee
0.4
2000
3000
4000
3000
4000
3000
4000
q R Knee
3.5
4 3.5
0.3
0.3
3
0.2
0.2
2.5
0.1
0.1
2
3 2.5 2 1.5 0
0
1000
2000
3000
4000
0
0
MN L Ankle
1000
2000
3000
4000
1.5
0
1000
MN R Ankle
0.8
0.8
0.6
0.6
2000
3000
4000
1
0
1000
q L Ankle
2000 q R Ankle
5
4.5
4.5 4 4 0.4
0.4
3.5
3.5
3 0.2
0.2
3 2.5
0
0
1000
2000
3000
4000
0
0
1000
2000
3000
4000
2
0
1000
2000
3000
4000
2.5
0
1000
2000
Figure 7-5: Results of validation test on the Central Pattern Generator: (a) Motoneuron activation (MN) in the muscles of the hip, knee and ankle joints (The solid line represents the extensor and the dotted line, the flexor.) (b) Joint angle displacements (q) of the right and left hip, knee and ankle joints the joint followed by the expected passive flexion of the joint due to gravity. Although the sensory stimulation only lasts for about 50ms, the extension of the joint angle takes about 300ms, which is due to inertia.
7.3.3
Golgi Tendon Organ Tests
There are two separate Golgi pathways which are differentiated by their functional characteristics as mentioned in section 7.2.1. The first set of pathways represents the positive force feedback loops which are exhibited during locomotor activity (Prochazka 1997). To validate these pathways, as an initial condition the body was suspended, and the left thigh was flexed at 90 . All the joints, except for the left knee were fixed. The knee extensor and flexor muscles were at rest. After 100ms, a mild perturbation of 100N was applied to the extensor, which was sustained for 500 ms (Fig. 7-7(a)). Due to this perturbation, the Golgi tendon organ receptor responds with an increase in activitiy (Fig. 7-7(b)). This provokes the reflex, which activates the motor neuron (Fig. 7-7(c)), causing muscle activation (Fig 7-7(d)). This again results in further Golgi sensor activation, as expected in a positive feedback loop. However
94
10
0.48 1 Knee Torque (Nm)
0
0.46 MN activation
Muscle length (m)
0.8
0.44 0.42
0.6 0.4 0.2
0.4
−10 −20 −30 −40
0 0.38
0
20
40
60
80
100
120
140
160
180
0
200
20
40
60
80
100
120
140
160
180
−50
200
0
200
400
600
800
1000
1200
0
200
400
600 time (msec)
800
1000
1200
1.75
8
4 2 0
0.8
Knee angle (rad)
Spindle Activation
Muscle velocity (m/s)
1 6
0.6 0.4 0.2
1.7
1.65
1.6
0 −2
0
20
40
60
80
100 time (msec)
120
140
160
180
200
0
20
40
60
80
100 120 time (msec)
140
160
180
200
1.55
Figure 7-6: Results of validation test on the muscle spindle related pathways of the knee joint: (a) muscle length (top left) and muscle velocity (bottom left) (b) motor neuron activation (top middle) and muscle spindle receptor activation (bottom middle) (c) joint torque (top right) and joint angle (bottom right) after approximately 300 ms, it can be noticed that the increase in golgi sensor, motor neuron and muscle activation begin to stabilize, due to the non-linear properties of the muscle. This is as predicted in experiments with human subjects (Prochazka 1997). The second set of pathways represent the negative feedback loops in response to extreme conditions of muscle tension. To validate these pathways, as an initial condition, the body was suspended and the left thigh is flexed to 90 ; all the joints are fixed, except for the left knee. The knee extensor muscle (Vastus Medialis) is fully activated. A perturbation is applied in which the knee extensor muscle is subjected to a strong force (150% of the maximum force that can be actively generated by the muscle), starting at 100 msec and lasting for 500 msec. In the first time step, it is observed that the Motoneuron activation goes from 0 to 1. This causes a smooth and delayed muscle activation, due to ion propagation model. The Golgi receptor senses the increased muscle force and it reaches the value of 0.5. The perturbation starts after 100 msec, when the muscle is fully activated; the Golgi sensor reacts to the strong force applied to the muscle, and fires at maximum rate. This provokes the reflex, which inhibits the Motoneuron, causing the delayed decrease in muscle activation, and consequently in the muscle force generated. After approximately 170 msec the receptor activity is decreased, according to the decrease of the muscle force. When the muscle is fully inhibited, the Golgi sensor firing rate reaches the steady state of 0.7. When the force of perturbation is released after 500 msec, the receptor stops firing and thus the reflex is no longer active. This response is as described in [91].
7.3.4
Cutaneous Pathway Tests
In the validation test of the cutaneous pathways, as an initial condition, the body is suspended, and all the joints are free. Due to the passive muscular forces, it reaches the equilibrium position with hip and knee slightly flexed and ankle slightly extended. Then, the left cutaneous receptor is stimulated for 50 msec till it reaches the firing rate value of 90%. As expected, we observe a strong and rapid activation of the Motoneurons of the exten95
MN Activation Force of Perturbation (N)
1500 1000 500 0
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500 600 time (msec)
700
800
900
1000
1 0.5
Muscle Activation
0 1
Golgi Sensor Activation
0.5 0 1
0.5
0
Figure 7-7: Results of validation tests on the Golgi tendon organ related pathways of the knee extensor: (a) Force of perturbation applied to evoke the reflex response (top) (b) motor neuron activation of the muscle (second from top) (c) muscle activation (third from top) (d) Golgi sensor activation (bottom) sors of the ipsilateral leg, and a weaker and slower activation of the flexors of the contralateral leg. This produces muscle torques that extend all the joints of the left leg and slightly flex all the joints of the right leg. The tests resemble the results shown in (Schieppati 1994) in which cutaneous stimulation was observed to produce facilitation of the extensor motoneurons in cats.
7.4
Overall Model Behavior
After the individual validation tests of the modalities were performed as described above, the body was placed on the treadmill, with 60% body unloading and all neural pathways were included into a single model. This gave rise to the issue of determining the appropriate relative strengths of the pathways with respect to each other. At the offset, the relative strength was conceptualized as a “knob” which could influence the gain of a particular category of reflex pathway by a positive multiplicative constant. This knob would alter the amplitude but not the overall function of the reflex pathway. By using four such knobs, ÃHM49E , ÃHÏ%< , ÃHE ! and Ã" ! , the relative strengths of the CPGs, muscle spindle pathways, golgi tendon pathways, and cutaneous pathways could be controlled respectively. It was decided that the CPG should be fully active during walking, that is, the gain should be 1. Preliminary tests with the gains of the reflex pathways indicated that the model performed well when these gains were lower than the CPG gain, but non-zero. The following results indicate, for example, stable walking behavior acheived on the treadmill when the pathway gains are set to [ Ã"M49Eei Q , Ã"Ï%< i &>'Q , ÃHE !i &>'¤Q , ÃH !i &>'P+ ].Using these values, the system was able to acheive a stable gait cycle with a walking speed of 0.345 m/s at a rate of 2.2 steps per second. Fig. 7-10 plots the joint angle trajectories for four steps, Fig. 7-11 plots the joint angle vs. angular velocity for 10s of the walking cycle, and the Fig. 7-12 96
1000 500
MN Activation
Force of Perturbation (N)
1500
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500 600 time (msec)
700
800
900
1000
0.5 0
Golgi Sensor Activation Muscle Activation
0
1
1
0.5
0 1
0.5
0
Figure 7-8: Results of validation tests on the Golgi tendon organ related pathways of the knee extensor: (a) Force of perturbation applied to evoke the reflex response (top) (b) motor neuron activation of the muscle (second from top) (c) muscle activation (third from top) (d) Golgi sensor activation (bottom) visually represents the gait cycle over a 2 sec time interval. The results indicate that it is indeed possible to acheive a walking gait cycle by integrating central pattern generators with proprioceptive and cutaneous reflex pathways known from human neurophysiology to be involved in gait generation.
7.5
Experiments and Clinical Relevance
While the model is able to successfully generate a walking gait pattern, this is not its most significant scientific or clinical contribution. The relevance of such a model lies in the new types of scientific investigations which it enables, as compared to previous models. For example, in previous work, clinical studies were performed investigating cutaneous and proprioceptive reflexes independently during walking. However, very few could focus on the issue of the relative criticality of the reflexes for walking. It was not known whether the cutaneous reflexes play a larger role than the proprioceptive reflexes or vice versa. It was also not known whether in the realm of proprioceptive sensing, the muscle spindle related pathways were more important than the golgi tendon organ related reflexes. This information, would be crucial for gaining a deep understanding of the mechanisms controlling walking, not only at the lower levels but at the higher cortical levels as well. For example, if force sensing were much more important than position sensing, we could draw the hypothesis that force-control related representation of motor movements exist in the motor cortex. Understanding the relative criticality of the reflex modalities would prove beneficial in therapy. State-of-the-art therapy for spinal cord injuries involve manual and orthosis-driven treadmill training moving the patients legs through an approximate gait pattern. However, if it was known for example that one of the reflex pathways was much more important in 97
1
0.5 0
0
50
100
150
200
1
0.5 0
0
50
100
150
200
50
100 time (msec)
150
100
150
200
−10
0.5
20
0
0
50
100
150
200
1
8 100 time (msec)
150
200
200
200
300
0
100
200
300
10 5 0 −5 −10
400
0
0
50
100
−10 −20
0
0
10
0.5
0.5
0
50
0
1
1
0
0
Hip Torque (Nm)
200
10
Knee Torque (Nm)
150
RIGHT LEG 15
0
100
200
300
400
0
100
200
300
400
0
100
200 time (msec)
300
400
10 0 −10
400 8 Ankle Torque (Nm)
100
0
Hip Torque (Nm)
50
0.5
Knee Torque (Nm)
0
LEFT LEG 20
1
Ankle Torque (Nm)
0
Ankle MN Activation Knee MN Activation
Hip MN Activation
RIGHT LEG
0.5
Left Cutaneous Ankle MN Activation Knee MN Activation Receptor Activation
Hip MN Activation
LEFT LEG 1
6 4 2 0 −2 0
100
200 time (msec)
300
400
6 4 2 0 −2
Figure 7-9: Results of validation test of the cutaneous pathways, when stimulation is applied to the left foot: (a) activation of extensor muscles of the left leg (first column from left) (b) light activation of the flexor muscles of the right leg (second column) (c) Joint torques of the left leg (third column) (d) Joint torques of the right leg (fourth column) walking than others, then specific therapies targeting those reflex pathways could be added to the therapy regime. In contrast to the other investigative techniques, the Neuro-musculo-skeletal model described is well suited to the investigation of relative criticality. The following set of experiments demonstrates the power of this approach, as well as presenting some insightful preliminary results.
7.5.1
Experiments
In the control condition the central pattern generator gain ÃHM49E was set to 1.0, and the muscle spindle pathway gains the Ã"Ï%< , golgi tendon organ pathway gains ÃHE ! and cutaneous pathway gains Ã" ! were set to 0.1. This condition was selected as it was a condition in which all reflex pathways were equally activated and the robot could produce stable walking behavior. This provided a suitable basis for observing changes in gait quality due to variation of system parameters. The focus of the first study was to investigate the significance of each reflex modality in the system. This was done by recording the effects of deactivation on system behavior. Three experimental conditions were tested as shown in Table 7.4. In the first experiment, the muscle spindle pathway gains were reduced to 0.0, eliminating the effect of these pathways from the system. In the second experiment, the golgi tendon organ pathway gains were reduced to 0.0, and in the third, the cutaneous pathways gains. The results of these experiments are given in Table 7.5. In each case the performance of the system was drastically reduced from stable walking to stumbling forwards and falling. This strongly indicated that each of the reflex modalities was playing a significant role in the successful performance of the system in the control condition, although the reflex gains were low.
98
Table 7.4: Experimental Conditions in Deactivation Study: In condition 1 the muscle spindle related reflex pathways are deactivated by setting the gain to 0, in condition 2 the golgi tendon organ related pathways are similarly deactivated, and in condition 3 the cutaneous reflex pathways gains. Ã"M49E
Ã"Ï%< ÃHE !
ÃH !
1
1
0.0 0.1 0.1
2
1
0.1 0.0 0.1
3
1
0.1 0.1 0.0
Table 7.5: Results of Deactivation Studies MS GT CT Duration Steps Speed Behavior (s) (m/s) 0.0 0.1 0.1 3.6 5 0.05 almost unable to step, falls forwards 0.1 0.0 0.1 3.9 6 0.08 almost unable to step, falls forwards 0.1 0.1 0.0 6.6 5 -0.10 almost unable to step, falls forwards
99
q L Hip
q R Hip
4
3.8
3.8
3.6
3.6 3.4 3.4 3.2
3.2 3 5000
6000
7000
8000
9000
3 5000
6000
q L Knee 3.5
3.5
3
3
2.5
2.5
2 5000
6000
7000
7000
8000
9000
8000
9000
8000
9000
q R Knee
8000
9000
2 5000
6000
q L Ankle
7000 q R Ankle
3.8
4 3.8
3.6
3.6 3.4 3.4 3.2 3 5000
3.2
6000
7000
8000
9000
3 5000
6000
7000
Figure 7-10: Joint angle trajectories when the biped has acheived a stable limit cycle, plotted for 4s. The focus of the second study was to gain insight into the relative criticality of the different reflex modalities. The control case, had the CPG fully active (gain = 1), and all the reflexes set to gain = 0.1. In the first experimental set, the cutaneous pathway gain was altered (gain = 0.5, 0.9), in the second, the golgi tendon organ pathway, (gain = 0.5, 0.9) and in the third the muscle spindle pathways (gain = 0.5, 0.9). The values 0.1, 0.5 and 0.9 were chosen to in order to sample the space at a low, mid-range, and high value of gain, while remaining lower than the CPG. The experimental conditions are summarized in Table 7.6. If better walking was achieved at a high level of gain than a lower gain, it would indicate that the modality was critical for walking. On the other hand, if the walking was better at a lower gain than at a higher gain, it would indicate that the reflex interfered with walking during certain parts of the gait cycle, and would likely require reflex modulation or be turned off completely during the gait cycle. Finally, if the walking was unaffected by the level of gain, then it could be concluded that the modality was relatively insignificant for walking. In each experiment, the body was started from the same initial conditions on a treadmill with the same speed. All other parameters, (excluding the reflex pathway gains) were also equal. An experiment was allowed to continue until the waist height had decreased past a threshold value of 0.45, which was approximately the length of the shank. This indicated that the waist had fallen past the knees, i.e. past the point of being able to recover a stable gait. Thus, the duration of the experiment was directly correlated to the quality of the gait, and could be used as metric for comparison. In addition to the quantitative data, qualitative information about the quality of the gait cycle was also relevant. Both these quantitative and qualitative results of the experiments are presented in Table 7.7. 100
Table 7.6: Experimental Conditions in Gain Modulation Study: In Set 1 muscle spindle related reflex pathway gains are altered, in Set 2 the golgi tendon organ related pathway gains, and in Set 3 the cutaneous reflex pathway gains. The line in bold indicates the control condition, which is identical over all sets. ÃHM49E
Set 1
Set 2
Set 3
1 1 1 1 1 1 1 1 1
ÃHÏ%< Ã"E !
ÃH !
0.1 0.5 0.9 0.1 0.1 0.1 0.1 0.1 0.1
0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.5 0.9
0.1 0.1 0.1 0.1 0.5 0.9 0.1 0.1 0.1
Table 7.7: Results of Gain Modulation Studies MS GT CT Duration Steps Speed Behavior (s) (m/s) 0.1 0.1 0.1 50.0 48 0.08 stable stepping, very low velocity 0.5 0.1 0.1 3.1 6 0.24 almost unable to step, falls forwards 0.9 0.1 0.1 3.9 5 0.14 almost unable to step, falls forwards 0.1 0.1 0.1 50.0 48 0.08 stable stepping, very low velocity 0.1 0.5 0.1 50.0 53 0.10 stable stepping, very low velocity 0.1 0.9 0.1 38.1 41 0.19 stable stepping, low velocity 0.1 0.1 0.1 50.0 48 0.08 stable stepping, very low velocity 0.1 0.1 0.5 9.6 11 0.44 walking forwards, falling after several steps 0.1 0.1 0.9 9.1 10 0.54 walking forwards, falling after several steps
101
q L Hip
q R Hip
4
4 2
2
0 0 −2 −2
−4
−4 2.5
3
3.5
−6
4
3
3.2
q L Knee
3.4
3.6
3.8
3
3.5
3.6
3.8
q R Knee
20
15
15
10
10
5
5 0
0
−5
−5 −10 1.5
2
2.5
3
−10 1.5
3.5
2
q L Ankle 200
200
100
100
0
0
−100
−100
−200
−200
−300 −400
2.5 q R Ankle
−300 3
3.2
3.4
3.6
3.8
−400
3
3.2
3.4
Figure 7-11: Phase plots of joint angle vs. angular velocity, plotted for 10s.
Figure 7-12: Gait pattern after the biped has reached a stable limit cycle, shown for 2s In the control condition, stable stepping was acheived but the velocity was low so the behavior almost resembled stepping in place. In the first set of experiments where the muscle spindle gain was altered, it was observed that at both mid-range and high values of gain, the walking was also strongly impaired; the biped was able to take only 5-6 steps before falling forwards. A major contribution to the fall was the buckling of the knees during stance phase as can be seen in Fig. 7-14. It appeared as though the flexors of the knees which were stretched during stance phase, produced too strong an opposing contraction causing the knee to flex and buckle during the stance. In the second set of experiments, the results were interesting. At both mid-range and high values of gain, it was found that within the range of gains chosen, no significant difference was produced in the gait pattern compared to the control condition (Fig 7-15). The data does not necessarily suggest that force dependent reflexes do not play a significant role during walking. However, they do suggest that within a certain range, positive force feed-
102
Figure 7-13: Movement pattern achieved in the control condition [ ÃHÏ%'¤Q , ÃH !óiÉ&>'Q ]
Figure 7-14: Movement pattern acheived in the case [ ÃHÏ%<Öi¿&>'@( , ÃHE !
ik&>'Q
ÃHE !
iÄ&>'¤Q
,
, Ã" !íi¿&>'¤Q ]
back does not decrease the stability of the system, as suggested by results from Prochazka. In the third set of experiments, in which the cutaneous pathway gains were altered, it was found that although the stability of the gait was affected for the worse, the quality of the gait improved. In the control condition, stable stepping was observed, but since the step length was quite small, the gait almost resembled stepping in place, causing the body to move backwards along with the treadmill. With an increased value of cutaneous reflex pathway gain, the gait matched the speed of the treadmill more closely and was able to take on the order of 10-20 steps before starting to walk forward and fall over. In the case of the mid-range gain, the speed acheived was 0.44 m/s, and in the case of the high gain the speed acheived was 0.54 m/s. Thus, considering in addition, the results of the overall test, where a cutaneous pathway gain of ÃH ! ik&>'@+ produced a walking speed of 0.345 m/s, it is possible to observe a monotically increasing relationship between the cutaneous pathway gain and walking speed.
7.6
Discussion
The results of the experiments provide interesting preliminary insights into the relative criticality of the different reflex pathways. It seems that the muscle spindle based stretch reflexes must remain at a low value of gain during walking, in order to enable stepping. The gain of the golgi tendon organ based reflexes do not seem very critical within a certain range. The cutaneous reflex gain on the other hand is observed to be correlated to the speed of walking. While further experiments, both in simulation as well as with human subjects, are required to more strongly qualify these results, the experiments serve to the highlight the 103
Figure 7-15: Movement pattern acheived in the case [ ÃHÏ%<Öi¿&>'¤Q , ÃHE !
ik&>')(
, Ã" !íi¿&>'¤Q ]
Figure 7-16: Movement pattern achieved in the case [ ÃHÏ%<Öi¿&>'¤Q , ÃHE !
ik&>'Q
, Ã" !íi¿&>'@( ]
potential of the neuromusculoskeletal model for the investigation of such issues. In addition to further sampling the space of reflex pathway gains by modality as performed above, various other studies can be undertaken. Studies can be performed of the relative strength of reflex gains by joint. It is likely that some reflexes are more strongly activated in some joints than in others, depending on their utility during the gait cycle. By sampling the space of relative reflex gains by joint, using an optimization algorithm, the relative gains which lead to successful gait could be determined. The model could also be used to address the more complex issue of temporal reflex gain modulation, using a similar approach. Another area of interest, which has been difficult to address using other techniques is that of reflex pathway interconnectivity. It is known that in the case of the proprioceptive reflex pathways, a sensory stimulus from one muscle not only activates the reflex in the homonymous muscle but also other muscles which may or may not be in the same muscle group. The topology of these interconnections is not entirely clear. Using the model however, it would be possible to assess the effect of an interconnection on the resulting quality of gait, and therefore determine likely interconnection schemes. Another important issue that can be addressed using the model is the relationship between the CPG and the reflexes. On one hand simple questions regarding the relative gains of the CPG vs. the reflexes can be addressed, using a similar approach to the experiments above. However, more complex issues such as the interconnections between the reflexes and the CPG can also be addressed. For example, it is known that the cutaneous reflex pathway is responsible for resetting the central pattern generator from swing phase to stance phase. On the other hand, the joint proprioceptors are responsible for sensing the end of stance phase, and for initiating the swing phase. These interdependencies can easily be investigated in the model by introducing the hypothesized connections between the proposed reflex pathways and the CPG.
104
Very few models have focused on two or more reflex modalities simultaneously and their inter-relationships during the walking cycle. Thus, the function of these reflex pathways are understood only in isolation, with respect to the other parts of the system. This leads to a bias in current literature towards attributing a direct gait related function to each reflex. However, studies of behavior-based systems which are composed of multiple reflexes, have shown that in a system of reflexes, some reflexes may serve to simply create the right conditions to activate other, more significant, reflexes. Therefore the function of a reflex may not directly relate to a key transition in the gait cycle, but may induce a small change which then triggers another reflex causing a key transition. This complexity has been largely ignored in the biological literature to date. The possibility to address these questions makes the model versatile in its range of potential applications. However, there are a few areas which can benefit from future work. Currently the model uses only monoarticular muscles. While the function of biarticular muscles are mostly redundant, the model could nonetheless be improved in accuracy by the addition of biarticular muscles. Another issue is that the model currently does not have any connections between the Ib interneurons across joints, and from the muscle spindle sensors. Both these kinds of connections are known to exist although their exact topology is not known (Kandel 1985). It will be interesting in a next step to incorporate these connections after validating them against relevant clinical data. Finally, the model could be validated further by ensuring that the behavior of the system does not change significantly when the appropriate neural latencies of the various reflex pathways are implemented as known from biology.
7.6.1
Topological vs. Functional Modelling
The main difference between the methodology used in this work, compared to previous work, is that we have used a largely topological modelling strategy as opposed to a functional modelling approach. In a functional modelling approach, the function of a certain neural module is first modelled as an abstract mathematical function or artificial neural network whose structure is an abstraction from the actual architecture of the neural pathways in biology. The quality of such a model is judged by comparing its functional performance to the known function of the neural module. If the model replicates the overall behavior of the system sufficiently well, it can be said that the model represents a viable hypothesis about the mechanisms involved in the system. In a topological modelling approach however, the aim is not to suggest a hypothesis about the mechanisms involved in the system, but to understand how known neural pathways contribute to the overall functioning of the system. Thus, a neural module is implemented according to its known topology, and evaluated by comparing its functional performance against its known immediate function. By putting this together with other pathways, and observing the emergent pattern of behaviors, the more indirect contributions of the module in the context of the complex system can be assessed. Each modelling approach has its own advantages and disadvantages, and is more or less suited to a particular type of scientific enquiry. The functional modelling approach has the advantage that it can lead to simple models, with explanatory power over a complex system. The disadvantage is that there can be several functional models which can produce 105
the same overall behavior, and it is difficult to decide which one most accurately represents the real system. A functional model is most suitable for understanding systems in which the overall behavior is easy to measure, but the neural pathways have not been well described, or a system in which the pathways have been well-described but an abstract level of understanding about the system is desired. The topological modelling approach on the other hand, is appropriate when the topology of a module is well known, but where all the functional contributions of that module may not be fully understood. There are, of course, certain disadvantages of the topological modelling approach as well. Firstly, using the topological approach for the modelling of a complex system can be cumbersome. The number of neural units and parameters can quickly escalate into the thousands, making the system large and slow. The model as it stands now, for example, already has over 300 nodes and 600 synaptic connections. Secondly, in topological modelling there is always the question “Is there enough detail?” For example, it can be argued that a more detailed neural model should be used, which includes temporal characteristics such as spiking. The problem is that it is difficult to assess when the level of accuracy is appropriate. The only way to do this would be to include the feature under debate, and note whether it effects a significant change in the behavioral output of the system. If the change is large, then the feature is important for the model. Unfortunately, this is a cumbersome task to undertake for every feature that can possibly be included in the system. Finally, topological modelling is limited by the current knowledge of neural pathways in biology. Only neural modules whose topology is relatively well known can be included in a topological model. Thus, it can be argued that topological modelling cannot provide any new insights, in addition to what is already known in biology. However, as shown above, there is some utility of a topological model as compared to a functional model. Although the topology of a particular neural pathway may be known, along with perhaps its direct function, its multiple range of functions in the system may not be known. By putting together known pathways, and seeing how they interact to produce a particular behavior, the functional contributions of that module may be more deeply understood. For example, it has been previously known that cutaneous pathways are crucial for the control of stance phase, but it was not known that the pathway may be involved in the regulation of walking speed. In this way, even a topological model with limited accuracy can generate new insights into the functional contributions of existing structures.
7.6.2
Clinical Application
In addition to these theoretical contributions, the model also has applications in clinical practice for the rehabilitation of spinal cord injured patients. Incomplete spinal cord injury, involving a partial lesion of the neural pathways of the spinal cord, can lead to various locomotor deficits depending on the location and size of the lesion. However, for a clinician with access to only the external pathologies of the patient, it is difficult to determine the exact nature of the internal lesion, and thus to prescribe an optimal therapy regime. Use of the neuro-musculo-skeletal model can help greatly in this respect. A host of different lesions can be performed on the model, and the resultant gait pattern of the biped on the treadmill can be recorded in a database. Then when a patient’s gait pattern is recorded, statistical methods can be used to determine the pattern of lesions which has the closest fit to 106
the patient’s condition. A more advanced method can be to develop a “reverse engineering” algorithm for use with the model, which enables the settings of gain and lesion parameters to be determined for any arbitrary gait pattern, providing invaluable diagnostic information to the clinician.
7.7
Conclusion
This paper describes the design and implementation of a biomimetic human neuromuskuloskeletal model, which is composed of a neural spinal cord model with high topological accuracy, and a biomechanical human body model. The neural spinal cord model contains neural modules corresponding to central pattern generators and muscle spindle, golgi tendon organ and cutaneous sensor based reflex pathways. Each of the subcomponents of the model are independently validated before inclusion into the overall model, and the system is shown to produce stable walking. The scientific potential of such a topological model is demonstrated in the experiments related to relative criticality of reflex pathways. The results provide interesting insights into the importance and function of the various reflex pathways in walking.
107
Chapter 8 Sensorimotor Control Sensorimotor Control of Biped Locomotion1 Human locomotion is extremely adaptive and robust to changes in environment, body size and morphology. The control of this highly adaptive system is thought to be performed by central pattern generators, supported by a system of sensorimotor connections called reflexes. Several studies on central pattern generators have been conducted, alone and in concert with reflexes. However, controllers based on direct sensorimotor reflexes have not been given much attention, as they have probably been considered too simple for the task of controlling a biped. In this paper, however, we present results to the contrary. It is shown that a purely sensorimotor neural network, that is one with direct connections between sensors and motors can produce stable gait in an 8 degree-of-freedom simulated lower body biped. Further analysis indicates that proprioceptive information is useful but not essential for the control of walking, while cutaneous information is essential. The final experiments show that sensorimotor control of biped walking can be accomplished using a network with only two cutaneous inputs and four neurons. The results suggest that reflexes may play a large role in rhythm generation, in addition to central pattern generators. The results also demonstrate the importance of sensorimotor reflexes in adaptive behavior.
8.1
Introduction
Human locomotion is well known for its highly adaptive and robust nature, in the face of external and internal changes. However, the mechanisms involved in the neural control of locomotion are not yet fully understood. There is some evidence which suggests that isolated neural circuits called central pattern generators exist in vertebrates to produce rhythmic oscillatory patterns, which coupled with the dynamics of the body produce locomotion. It has been shown that even in the complete absence of stimulus these circuits in the spinal cord are capable of producing oscillatory motor outputs [33]. Experiments with cats have also shown that a decerebrated cat can walk on a treadmill when some parts of the brain stem are stimulated [107]. Such capacity for autonomous rhythm generation 1
Paul, C. (2004) “Sensorimotor Control of Biped Locomotion”, accepted to Journal of Adaptive Behavior
108
has also been observed in humans. In experiments with complete adult paraplegic patients with essentially transected spinal cords, it is shown that they are able of producing stepping movements when supported on a moving treadmill [20]. Also, in experiments with newborn infants it has been shown that they are capable of producing stepping movements when they are held upright [114]. These results suggest that the same neural circuits which are capable of rhythm generation in vertebrates may also be involved in human locomotion. Computational studies have shown that small neural circuits consisting of 2-6 mutually inhibiting neurons are capable of such autonomous rhythm generation [67][68]. The half-center hypothesis of central pattern generation [10], is however more prevalent for vertebrates. This proposes that small neural units of two reciprocally inhibited “half-center” neurons are responsible for rhythm generation in each joint. The half-center model of a CPG has been implemented by Matsuoka as a set of differential equations [67], which has been widely used in studies of rhythmic movement generation. Kimura et al [57] developed a neural controller for a quadruped robot, while Taga[111] and Miyakoshi et al [74] have used the Matsuoka oscillator in a similar neural controller for biped locomotion. These studies all support the hypothesis that central pattern generators are important for rhythm generation in walking. However, biological results also suggest that reflexes play a large role in walking. For example, the flexor reflex afferents are known to be responsible for resetting the rhythm produced by the central pattern generators in cat, measured during fictive locomotion [104]. The cutaneous reflex is also known to play a large role in some phases of the gait cycle, and has been found to be finely modulated during locomotion [18] [105]. Experiments have shown that these afferent pathways are also involved in regulation of rhythm as they are responsible for appropriately inhibiting the flexor and activating the extensor for switching from swing to stance phase [23]. Furthermore, the proprioceptive reflex pathways, are responsible for switching from stance back to swing phase [81]. Thus, there is ample evidence that reflexes are largely involved in regulation of walking rhythm. Despite these findings, however, very little attention has been paid to the role of direct sensorimotor connections with respect to rhythm, in simulated studies of walking as well as in real world biped robots. In simulation, studies by Hase et al [37], Ogihara et al[?] and Rybak et al [102] have included reflexes in addition to CPGs in their neural control systems, but as the gait is optimized by evolving all the parameters of the network simultaneously, the independent contribution of the reflexes cannot be distinguished. The only study which has investigated the role of purely sensorimotor control in walking has been conducted by Wisse et al [120] In their work, they have shown that a 4 DOF lower body biped, can be controlled to walk in 2D on level ground using only reflexes based on the foot contact sensors. In this work we sought to further understand the role of sensorimotor connections in walking, by using a sensorimotor network with multiple sensory modalities to control a biped of greater complexity. Our biped has 8 DOFs, with one DOF at each ankle, one at each knee, and two at each hip, enabling walking in 3D. A genetic algorithm was used to optimize the neural controller as in some previous studies [27][98][24][25]. However, the neural controller used was simply a one-layer neural network, with sensory inputs from the biped body connected to motoneuron outputs which commanded the joints of the biped. It was hypothesized that although stable walking may not be observed in this system, partial 109
results such as appropriate switching from swing to stance phase or vice versa could be observed, providing insight into the role of sensorimotor mechanisms in bipedal walking.
8.2
Biped Morphology HR
HR
HP
HP
K
K
A
A
z
x y
Figure 8-1: The biped morphology: The lower body biped is a 7-link structure with 8 degrees of freedom. Each hip, knee and ankle joint has one degree of freedom in the sagittal plane. Additionally each hip joint has one degree of freedom in the frontal plane. The robot is a 7-link biped with 8 degrees of freedom (Fig. 8-1), simulated in a realtime, physics-based virtual environment2. The robot has a waist, two upper leg, two lower leg, and two foot links. Each knee and ankle joint has one degree of freedom in the sagittal plane. Each hip joint, connecting the upper leg to the waist, has two degrees of freedom: one in the sagittal plane and one in the frontal plane. These correspond to the pitch and roll motions. The joints are limited in their motion with joint stops. The hip roll joint on each side has a range of motion between and degrees with respect to the frontal plane. The hip pitch joint has a range of motion between and , with respect to the sagittal plane. The knee joint has a range of motion between and with respect to the axis of the upper leg link to which it is attached. Each of the joints is moved by a simulated torsional actuator. The actuator receives position commands from the controller. It uses proportional control to determine the velocity of the link, with a relatively low maximum torque ceiling. 2
MathEngine PLC, Oxford, UK, www.mathengine.com
110
The torque applied to actuators is determined by jm¨ ² ÒÂi
þ
}Hì $Z$K¨1ÞÃ $%Søp00gjpl y{zI0
(8.1)
where is the actual joint angle, Sø is the desired joint angle, jmlJy{z is the maximum torque ceiling, ei h , and is the feedback gain matrix. This means that the velocity of a link will be greater the further it is from the commanded joint angle position, but if the force required to achieve this velocity is too large, it will only apply the maximum force. This mechanism incorporates a measure of compliance into the system, and is in accordance with the capabilities of real world actuators. The link widths of the biped have the ability to change within a certain range. Their exact value is determined by the evolutionary algorithm. This enables the optimization algorithm to use morphology as an extra-dimensional bypass [4], enabling faster convergence. All the other physical parameters of the biped are static, as given in Table 1.
Table 8.1: Morphological parameters of the biped robot. A ul represents one unit length defined as the five times the radius of the spherical sockets at the hip and knees. A um represents one unit mass defined as the mass of the same spherical socket. The ranges of the parameters which vary under evolutionary control are shown in square brackets.
Index Object 1 2 3 4 5 6
Knee sockets Hip sockets Feet Lower Legs Upper Legs Waist
Index Joint 10 11 12 13
8.3
Ankle Knee Hip Hip
Dimensions
Position
Mass
r = 0.2 ul r = 0.2 ul r = 0.4 ul, w = 0.8 ul r = [0.04, 0.16] ul, h = 8 ul r = [0.04, 0.16] ul, h = 8 ul r = [0.04, 0.16] ul, w = 8 ul
x = ½ 4 ul, y = 0 z = 8.26 ul x = ½ 4 ul, y = 0, z = 16.26 ul x = ½ 4ul, y = 0.26 ul, z = 0.26 ul x = ½ 4 ul, y = 0, z = 4.26 ul x = ½ 4 ul, y =0, z = 12.26 ul x = 0, y = 0, z = 16.26 ul
1 um 1 um 1 um [0.1, 0.4] um [0.1, 0.4] um [0.1, 0.4] um
Plane of Rotation
Range of Motion Ø
sagittal sagittal sagittal frontal
Ø
& (radians) Û Û Ò ~ Ò 2 2
Sensorimotor Neural Network
The sensorimotor network (Fig. 8-2 ) is a bilaterally decoupled neural controller [80]. This means that each half of the controller is a rhythm generator which only receives sensory inputs from the sensors of one leg and global sensors, and controls the joints of the same leg. Thus, there is no explicit contralateral communication or coupling between the two 111
LEFT LEG CONTROLLER
RIGHT LEG CONTROLLER
INPUT LAYER
B1
INPUT LAYER WO WLF WH LR LP LK
HP
HR
K
LT
RT RK RP RR WH WRF WO
A
A
OUTPUT LAYER
K
HR
B1
HP
OUTPUT LAYER
Figure 8-2: The sensorimotor neural network controller: This is a bilaterally decoupled controller, each half of which only has direct connections between the sensors and the motors for each leg. The sensors of the input layer are labeled as follows. WO: waist orientation WLF: difference between waist sagittal position and left foot sagittal position WH: waist height LR: left roll LP: left pitch LK: left knee LT: left foot contact (or touch sensor) RK: right knee joint angle position RP: right hip pitch joint angle position RR: right roll joint angle position RT: right foot contact (or touch sensor) WRF: difference between waist sagittal position and right foot sagittal position. The output layer consists of the motoneurons for the hip pitch (HP), hip roll (HR), knee (K) and ankle (A) joints. halves of the controller. Each rhythm generator receives eight sensory inputs, each of whose values are scaled to the range ¸Q_gvQ . These inputs correspond to left (or right) joint angle positions (for hip roll, hip pitch, knee and ankle joints), left (or right) foot contact information, waist orientation in the transverse plane, the difference of waist sagittal position and left (or right) foot sagittal position, and waist height. The input layer also has one bias node which constantly emits a signal of Q . The selection of sensors was based on the successful network topology described in [80]. In that work it was established that a recurrent neural network with one hidden layer could successfully control the gait and direction of a 6 DOF lower body biped robot, if it received inputs from the sensors described above. The current sensorimotor network retains the structure of the input and output layers from that network, while removing the intermediate hidden layer to form a purely sensorimotor network. The output layer consists of motoneurons which control the left (or right) hip pitch, hip roll, knee and ankle joints. The activations of the output neurons are computed by
}i
where
Õ
w
Õ ZÒ
ªAÕ â!Õ
is the output of a neuron in the output layer. 112
(8.2) ªAÕ â
are the synaptic weights of the
connections between the sensory inputs in the input layer to the motoneurons of the output layer. The output of a motoneuron is then given by i
G Qbã
± y
·Q
(8.3)
The values at the output layer are scaled to fit the range of their corresponding joint’s range of motion. Torsion is then applied at each joint to attain the desired joint angle.
8.3.1
Initial Condition
At the beginning of each simulation all the joint angles and velocities of the biped are 0, and both foot contact sensors are 1. Thus, the sensory inputs of the right and left rhythm generators are identical. As the weights of the two rhythm generators are also identical, identical outputs are produced. This could only lead to two situations: standing in place or hopping, which are the two behaviors in which both legs perform identical movements. It would not lead to walking, as the start of walking is an asymmetric motion: one leg enters swing phase as the other one stays on the ground. Thus, the control of the start of walking must be performed externally to the network. In other words, the initial conditions must be determined so that the network would function. As the initial conditions are closely coupled to the performance of the network, which is determined by the weights set by the genetic algorithm, it was also necessary to let the genetic algorithm control the initial conditions. Thus, the initial motor commands for the actuators of the biped in the first 10 steps of the simulation are eight values generated by the genetic algorithm. The network takes control of the biped after this time has elapsed.
8.4
The Genetic Algorithm
A fixed length genetic algorithm was used to evolve the controllers. Each run of the genetic algorithm was conducted for 300 generations, using a population size of 200. At the end of each generation, the Q&_& most fit genomes were preserved; the others were deleted. Tournament selection with a tournament size of three, is employed to probabilistically select genotypes from among those remaining for mutation and crossover. G_( pairwise one-point crossings produce (*& new genotypes: the remaining (*& new genotypes are mutated copies of genotypes from the previous generation. The mutation rate was set to generate an average of mutations for each new genome created, where was defined as a function of the genome length "Ir , as i#"*r Q*Q . Mutation involved the replacement of a single value with a new random value. Each genome contains floating-point values which are rounded to two decimal places and range between ¸Q_'P&_& and Q_'P&_& . The genome encodes +HG synaptic weights of the neural network, + morphological parameters to determine the link widths, and ? initial position commands, and thus has a total genome length of OH+ parameters. During evolution each individual is evaluated for 2000 time steps of the dynamics simulation. The initial condition for each individual at the first time step is one in which all 113
joint angles and velocities are set to zero. This results in a fully upright posture, with all parts aligned in the frontal plane, and both feet at an equal distance from the target. The evaluation is prematurely terminated if the center of gravity of the waist drops below the original vertical position of its knees (it falls) or if it “twists” too much, or if both feet lift off the ground, (it starts to run). This third termination criteria was added because the primary interest of this project was to study walking, and not running gaits. At the end of the evaluation, the distance of the biped traveled in the sagittal plane (determined relative to its original position) was considered its fitness.
8.5
Results
The Reactive Network Two initial probe experiments were performed with the sensorimotor network. It was hypothesized that it may not be possible to evolve stable gait with only a reactive network, but perhaps that stepping behavior could be observed. If this was the case, the results could provide some explanation for the control of this particular subtask in walking. However, the results showed that one of the two bipeds had evolved a stable gait cycle. In order to determine that these results were accurate, a suite of 20 evolutionary experiments were performed, each started with a new random seed. The history of the highest fitness in each generation, for each of the experiments is charted in Fig 8-3(a). As mentioned above, the fitness is the distance traveled by the agent during the evaluation time. As can be seen, several of the bipeds in these experiments evolved stable gait cycles which enabled them to walk a considerable distance during the given evaluation time. These results clearly showed that it was not only possible to get stable walking with a purely sensorimotor network, but also quite statistically probable. Lesion Studies In light of the results of the previous experiments, it was necessary to look more closely at the network to discover why this network was able to control walking. The question was “Which sensory inputs are important for such sensorimotor regulation of gait?” To determine this, a series of lesion studies was envisioned, each of which targeted a specific sensory modality. The lesion studies were labeled L1 to L5. In L1 the foot contact sensors were selectively lesioned, in L2 the joint angle position sensors were lesioned, in L3 the waist height sensor, in L4 the waist orientation sensor and in L5 the waist-to-foot sagittal position sensor. In each lesion study, 20 evolutionary experiments were performed as above. The results of these experiments, plotting the highest fitness in each generation, are shown in Figure 8-3(b)-(f). As can be seen, the results for lesions L2-L5 look similar to the results of the full reactive network. However the results for lesion L1 are dramatically different. The highest fitness achieved in all 20 runs is drastically lower than in any of the other cases. A visual summary of these results is provided in Figure 8-4. In this figure, for each case study, the highest fitness in each generation has been averaged over the 20 experiments. H denotes the healthy sensorimotor network, while L1-L5 stand for the lesion studies as described above. From these graphs, it can be seen that the results of the lesion studies L2-L5 114
H: Best Fitness
L1: Best Fitness
40
45
35
40
35
30
30 Distance Travelled (ul)
Distance Travelled (ul)
25
20
15
25
20
15
10 10 5
5
0
−5
0
0
20
40
60
80
100 120 Generations
140
160
180
−5
200
0
20
40
60
80
L2: Best Fitness
100 120 Generations
140
160
180
200
140
160
180
200
140
160
180
200
L3: Best Fitness
35
45
40
30
35 25
Distance Travelled (ul)
Distance Travelled (ul)
30 20
15
10
25
20
15
10 5 5 0
−5
0
0
20
40
60
80
100 120 Generations
140
160
180
−5
200
0
20
40
60
80
L4: Best Fitness
100 120 Generations L5: Best Fitness
45
40
40
35
35
30
30 Distance Travelled (ul)
Distance Travelled (ul)
25 25
20
15
20
15
10 10 5
5
0
0
−5
0
20
40
60
80
100 120 Generations
140
160
180
200
−5
0
20
40
60
80
100 120 Generations
Figure 8-3: Fitness Graphs: The graphs plot the highest fitness achieved in each generation, for each experiment. H denotes experiments conducted with the healthy sensorimotor network (top left). L1 denotes a lesion of the cutaneous sensors (top right). L2 denotes a lesion of the joint angle sensors (middle left). L3 denotes a lesion of the waist height sensor (middle right). L4 denotes a lesion of the waist orientation sensor (bottom left) L5 denotes a lesion of the waist-to-foot sagittal position sensor (bottom right).
115
are in fact better than the results of the full reactive network. This was not quite expected, as it had been assumed, especially in the case of the joint angle sensors, that this information was essential for walking. Furthermore, the results of the lesion study L1 were also intriguing. While it had certainly been known that cutaneous information was important for walking, it had not been known that it was essential. Not even a single one of the 20 experiments produced results beyond stepping forward and falling over. Average Fitnesses 30
L3
25
L4 L2
Distance Travelled (ul)
20
L5
H
15
10
L1 5
0
−5
0
20
40
60
80
100 120 Generations
140
160
180
200
Figure 8-4: Average Fitness: Each of the curves represents the average of the highest fitness in each generation over 20 experiments. Graphs are shown for experimental conditions, H (solid line) and L1-L5 (dotted lines). Cutaneous Network Since the results of lesions L2-L5 were on average better than the reactive network, one interpretation was that the sensory information provided by these sensors was somehow interfering with the reactive control of walking, and thus removing them may lead to better results. The only sensor that seemed essential for the reactive control of walking was the cutaneous sensor, since without it the network could not function. Thus, somewhat skeptically, it was decided to lesion all the non-essential sensors at once (Fig. 8.5), and determine whether only the cutaneous sensors were sufficient to regulate a part of the walking cycle and if so, which part. (This experimental condition is referred to as CUT). In the results produced, it was observed that in most runs the highest fitness achieved was quite low, and did not lead to a gait cycle. However, in two of the twenty experiments walking had evolved (Fig 8-6). And upon evaluating these walkers with unlimited evaluation time, it was observed that they would continue walking for as long as the simulation was run, i.e. they were stable. These results were interesting given that there was only one sensor and four neurons per leg. A short movement sequence from the gait cycle of the best walker in this set of experiments, is shown in Figure 8-7 (a real-time movie is available at www.ifi.unizh.ch/ chan116
LEFT LEG CONTROLLER
RIGHT LEG CONTROLLER
INPUT LAYER
B1
HP
INPUT LAYER
LT
HR
K
B1
RT
A
A
OUTPUT LAYER
K
HR
HP
OUTPUT LAYER
Figure 8-5: Cutaneous Network: This is essentially the sensorimotor network with all sensory inputs, excluding cutaneous information, removed. The sensors of the input layer are LT: left foot contact (or touch sensor) and B1: bias node and RT: right foot contact (or touch sensor) and B1: bias node. As before, the output layer consists of the motoneurons for the hip pitch (HP), hip roll (HR), knee (K) and ankle (A) joints. dana/reactive/movie.mpg). In the top four images, the left leg is taking a step forward, and in the bottom four, the right leg is moving. It can be seen that although the swing leg does not perform a full swing it allows sufficient ground clearance for the foot to be transferred to a new location. The evolved synaptic weights of the network for this walker, are shown in Figure 88. In this network, essentially one set of joint angle positions is calculated for the stance phase and another for the swing phase. Merely switching between these two desired positions depending on the foot contact produces the stable gait observed in this walker. The switching of the desired joint angle setpoint can be observed in the motoneuron outputs of the network (Fig. 8-9). The resultant joint angles of the biped over time are plotted in Figure 8-10, and the phase plots of the gait cycle are shown in Figure 8-11.
8.6
Discussion
The results of the initial experiments with the full sensorimotor network (H) indicated that a purely sensorimotor network, was capable of producing stable gait in 3D in a lower body biped. Then, lesion studies L1-L5 were conducted on each sensory modality independently, which showed that while proprioceptive sensing related to joint angles and body orientation are useful, they are not essential for the sensorimotor control of walking, while the cutaneous sensors are essential. Finally, the last set of experiments revealed that sensorimotor control of walking can be achieved using only cutaneous sensing. The results run counter to the current understanding of bipedal locomotion, in which it is assumed that a control module with the intrinsic capacity for rhythm generation must exist in order to sustain rhythmic gait. The results of our experiments, show that this is not 117
FC: Best Fitness 25
20
Distance Travelled (ul)
15
10
5
0
−5
0
20
40
60
80
100 120 Generations
140
160
180
200
Figure 8-6: Fitness History (CUT): The graphs indicate the highest fitness in each generation, for each of the 20 experiments with the cutaneous network true. Rhythmicity can be generated in the absence of recurrent neural structures, by direct sensorimotor connections which induce cyclic dynamics in the interaction of the biped morphology with the environment. In our case, the cutaneous sensorimotor connections produce desired joint angle positions for swing and stance phases which induce dynamics such that the next foot contact condition, is almost contralaterally identical to the previous one. This ensures the repeatability of the motion, and thus gives rise to its cyclic nature. The results also serve to assign a relative priority to the various sensory modalities. While it has been known that proprioceptive as well as global orientation information is used in walking, it was not known that walking in a lower body biped could be achieved without them. Thus, it may be that the role of these sensory modalities have been overestimated, and that perhaps such information is not as important for the generation of rhythmic movement during walking, for the lower body. 3 Conversely, the possible role of cutaneous sensing with respect to rhythm generation has been largely underestimated. While it has been known that they play a role in the reg3
An aspect of these results, which may lead to some confusion is that the performance of the individual lesion studies L2-L5 was better than the full reactive network. At first, the simple deduction was that these sensors possibly interfered with the performance of the network and were better left out. But the results of the cutaneous network experiments show that this is not the case. If the network was indeed better off without all the other sensors, an improved average performance would have been observed, but this was not the case. One probable cause is that the some of the information from the different sensory channels are redundant. Thus, if two channels deliver the same information, the evolutionary optimization algorithm will have to tune two sets of synaptic weights with no additional benefits, which will take more time. If one of the channels is removed, the number of parameters to be tuned is reduced, and thus the algorithm converges faster. Another factor, which most likely acts in conjunction, is that the exclusion of various sensory information influences the fitness landscape in different ways, changing the path the algorithm can traverse through the landscape. Thus, although the final result of excluding a sensor may in fact be worse at the end when all the populations have converged, it could appear better along the way. The actual performance can only be determined by performing the experiments again such that they run until all the populations have converged. This may be an interesting direction of future work.
118
Figure 8-7: A short movement sequence depicting one step (CUT): In the top four images the left leg steps forward, and in the bottom four images, the right leg. LT / RT
−0.87
LT / RT
0.26 0.43
HP
B1
HR
0.4
0.4
K
B1
0.67
0.0
A
HP
HR
0.43
K
A
Figure 8-8: Evolved synaptic weights of the best walker (CUT) ulation of rhythm [23], it has not been known that they are capable of producing rhythmic gait in the absence of central rhythm generation. This new insight makes it necessary to reconsider the mechanisms of rhythm generation during walking, with particular focus on the role of the central pattern generator. Is the CPG a redundant structure? Or are the cutaneous sensorimotor pathways wired up so they perform a separate function and do not interfere with walking rhythm? It is not likely that the CPG is entirely redundant. For one, in bipedal locomotion controlled by a sensorimotor network, as described in this work, the speed can only be regulated by changing the synaptic weights. Although possible, it is not likely that the neural synaptic weights change every time a person speeds up or slows down. It is more likely that a neural structure which can change its frequency according to incoming neural activation, such as a central pattern generator, causes the fast changes in walking frequency and step length. 119
Motoneuron Outputs
Right Hip Pitch MN
0.4 0.2 0
−0.2 −0.4 1000
1100
1200
1300
1400
1500
1600
1700
1800
1900
2000
1600
1700
1800
1900
2000
1600
1700
1800
1900
2000
1600
1700
1800
1900
2000
Right Hip Roll MN
0.4 time 0.3 0.2 0.1 0 1000
1100
1200
1300
1400
time
1 Right Knee MN
1500
0
−1 −2 1000
1100
1200
1300
1400
time
0.5 Right Ankle MN
1500
0.45 0.4
0.35 0.3 1000
1100
1200
1300
1400
1500 time
Figure 8-9: Motoneuron Outputs (CUT): The graphs indicate the motoneuron outputs of the right hip pitch (top) hip roll (second from top), knee (third from top) and ankle joint (fourth from top) for 1000 time steps. However, if cutaneous information can play such an effective role in rhythm generation, would its utility have been ignored in evolution? And if it was not ignored, then how could the two mechanisms of rhythm generation be coordinated? Just to illustrate that this problem is not trivial, consider what happens when the central pattern generator is oscillating at a high frequency and switches the control of a swing leg to stance phase before the leg has touched the ground. This would cause the foot to hit the ground during swing, making the person to trip. However if the CPG must listen to the cutaneous sensors, then the cutaneous sensors are in effect responsible for the rhythm and the CPG is no longer in control of the frequency. How can it then be possible for them to coordinate in a meaningful way, while still allowing for changes in walking speed? There has not been enough experimental investigation of this issue yet to answer this question. However, in light of our results we propose a new hypothesis, which may be worth further investigation. 120
Left Hip Pitch
Right Hip Pitch
0.5
0.5
0
0
−0.5
−0.5
−1 1000
1200
1400 1600 Left Hip Roll
1800
−1 1000
2000
0.15
0.15
0.1
0.1
0.05
0.05
0
0
−0.05 1000
1200
1400 1600 Left Knee
1800
−0.05 1000
2000
1
1
0.8
0.8
0.6
0.6
0.4 1000
1200
1400 1600 Left Ankle
1800
0.4 1000
2000
0.4
0.4
0.35
0.35
0.3
0.3
0.25
0.25
0.2 1000
1200
1400
1600
1800
0.2 1000
2000
1200
1400 1600 Right Hip Roll
1800
2000
1200
1400 1600 Right Knee
1800
2000
1200
1400 1600 Right Ankle
1800
2000
1200
1400
1800
2000
1600
Figure 8-10: Joint angle positions over time (CUT): The graphs track the joint angle positions for the joints of the left and right legs over 1000 time steps
A new hypothesis Since the results show that the control of rhythmic gait in the lower body can be achieved using only cutaneous sensorimotor pathways, it may be that the CPG is actually responsible for controlling the upper body. The frontal inclination of the upper body influences the cutaneous load sensing on the feet. When the upper body is inclined to the left, the cutaneous sensor of the left foot is highly activated, while that of the right foot is low. Conversely, when the upper body is inclined to the right, the right foot cutaneous sensor is highly activated, and the left foot cutaneous sensor is low. If the cutaneous reflexes are active, then when upper body inclines to the the left activating the left foot sensor, the right leg will initiate swing phase. Then when the body inclination changes to the right, the right leg will hit the ground, activating the right foot sensor (and deactivating the left foot sensor), which will cause the left leg to initiate swing phase. Thus, the CPG could control the frequency of stepping by using the inclination of the upper body as an indirect means of communication 121
Left Hip Pitch
Right Hip Pitch
30
30
20
20
10
10
0
0
−10
−0.4
−0.2 0 Left Hip Roll
0.2
−10
0.4
1
1
0.5
0.5
0
0
−0.5
−0.5
−1 −0.02
0
0.02 0.04 Left Knee
0.06
−1 −0.02
0.08
4
4
2
2
0
0
−2
−2
−4 0.5
0.6
0.7 Left Ankle
0.8
−4 0.5
0.9
2
2
1
1
0
0
−1
−1
−2 0.24
0.26
0.28
0.3
0.32
−2 0.24
0.34
−0.4
−0.2 0 Right Hip Roll
0.2
0.4
0
0.02 0.04 Right Knee
0.06
0.08
0.6
0.26
0.7 Right Ankle
0.28
0.3
0.8
0.32
0.9
0.34
Figure 8-11: Joint angle phase plots (CUT): The graphs depict phase plots of joint angle vs. velocity over 1000 time steps. with the sensorimotor rhythm generators of the lower body.
8.7
Conclusion
To summarize, our results show the following for an 8 DOF lower body biped:
$ a purely sensorimotor neural network can control stable walking in 3D $ cutaneous sensor information is essential for the sensorimotor control of walking $ joint angle sensing, as well as orientation information, is not essential $ a sensorimotor network based only on cutaneous information can control stable walking in 3D 122
Thus, this paper shows for the first time that a purely sensorimotor network, that is a network with only direct connections between sensors and motor outputs, can produce stable 3D walking in a 8 DOF lower body biped. Furthermore, it shows that joint angle information, as well as other sensors, are not crucial for the generation of a rhythmic gait pattern, whereas cutaneous load sensing is essential. It has also been shown that a sensorimotor network based purely on cutaneous sensor information is able to generate stable walking in a lower body biped. The results shed new light on the possible role of the cutaneous reflex pathways in rhythm generation, and on a possibly altered role of the Central Pattern Generator in walking. While previously, it was assumed that the CPG is responsible for rhythm generation in the joints of the lower body, it has been shown in this paper that reflex pathways would be sufficient to accomplish this task. Thus, this leads to a new hypothesis that the role of the CPG may in fact be in regulating the movement of the upper body, which influences the cutaneous load sensor information indirectly inducing the rhythm generated by the reflex pathways.
123
Chapter 9 Discussion 9.1
Summary of Research
This thesis presents the results of research focused on investigating the interaction between the physical body and neural control in biped locomotion. Chapter 2 presents a simulation study of a lower body biped robot, in which the weights of a recurrent neural network controller, as well as the physical parameters of mass distribution of the robot, are simultaneously optimized by a genetic algorithm to achieve walking. The results show that as the evolutionary algorithm has greater control over the mass distribution, optimizing the control of walking becomes easier. Thus, it suggests that in a bipedal system which uses neural control with sensory feedback, the physical dynamics of the body strongly influence the behavioral outcome of the system, and can thus, be effectively utilized to optimize control. Chapter 3 presents a simulation study investigating the potential role of the body in “relaying” information between disconnected neural modules. A lower body biped is controlled by a bilaterally decoupled neural controller, that is a controller in which the control modules of the left and right sides of the body have no direct interconnections between them. Optimizing the control of walking using a genetic algorithm, it is shown that coordination between the two sides can be achieved due to the physical movements on one side of the body relaying information to the sensors of the other side, through the dynamics. Chapter 4 is an investigation of the potential role of the upper body in locomotion, through the development of the robot STUMPY. The robot STUMPY, which has two degrees of freedom corresponding to a waist joint and shoulder joint, is shown to be able to locomote using only the indirect motions induced in the lower body by the motions of the upper body. It demonstrates, that instead of the upper body merely being a load that is carried, its motions could contribute to locomotion through the dynamics of the body. Chapter 5 applies the principle of upper body actuation, as developed in the previous chapter, to the development of a full lower body biped robot. The BENDY robot is a biped
124
with 6 passive degrees-of-freedom in the lower body, and 5 in the upper body. The robot has only one actuated joint at the waist, which can move in the frontal plane. This joint is used to cause side-to-side oscillation of the upper body, which causes the legs to alternately lift off the ground and passively swing forward. The robot demonstrates how movement of the upper body can be used to aid the dynamics of a bipedal lower body in achieving locomotion. Chapter 6 focuses on the investigation of the neural pathways of the human spinal cord, and how they interact with the dynamics of the body to produce locomotion. It describes the development of a human neuro-musculo-skeletal model with a lower body musculo-skeletal system which is closely biomimetic, as well as a neural architecture including pathways related to central pattern generators and spinal reflexes closely mapped from neurophysiology. Experiments are performed with the model, to investigate the relative importance of the sensorimotor interactions induced by the reflexes in locomotion. The results show that the cutaneous reflex which conveys information about the body dynamics at foot contact, is important for walking. Chapter 7 describes a simulation study which investigates the role of sensorimotor pathways in isolation. A lower body biped robot is controlled by a one-layer neural network which connects joint angle and cutaneous sensors directly to output motoneurons. Optimizing the weights of the neural network using an evolutionary algorithm, it is found that walking can be produced in the biped only using cutaneous sensorimotor connections. The study shows that direct sensory motor connections can interact with the inherent dynamics of a bipedal morphology to produce rhythmic movements. Chapter 8 introduces the idea that the role of the body can be viewed from a computational perspective. Through a series of thought experiments, it proves that physical structures are capable of performing computations and then discusses the implications of this fact for the control of multi-link robots such as manipulators or bipeds. It thus provides a theoretical foundation for the experimental results of the previous chapters, which demonstrate the morphology and control trade-off in bipeds. The following sections 9.2-9.8 will present the main scientific contributions of these chapters, viewed as a collection of results. Section 9.9 will then describe the software tools and robot platforms which have emerged as products of this research. Section 9.10 will describe a personal experience which has relevance for this work. Section 9.11 will survey the implications of the results for rehabilitation of spinal cord injured patients and section 9.12 will conclude with a summary.
125
9.2
Role of the Body in Neural Control
Chapters: 2, 3, 4 Morphological Optimization in Evolution As the results of Chapter 2 showed, a small change in the morphology can lead to a large improvement in performance. One implication of this result is clear: natural selection has been able to use this fact to fine tune the morphology of the human body to optimize the performance of frequently performed motor tasks such as walking and running, which are relevant for survival. In addition to this direct implication however, the results also provide further insights into the principles underlying the interaction between the body and neural control which are not as self-evident. The results showed that the agents which had achieved the best performance in all three conditions of mass distribution, arrived at a morphological solution in which the upper legs and waist were heavier than the lower legs. This is an interesting result because it resembles the biological situation; the human morphology also has upper legs which are more massive than the lower legs. The common understanding for this feature has been that the muscles of the upper leg are required to exert greater torques than those of the lower legs and must therefore be larger and stronger. However, in the experiments described the mass distribution bore no correlation to the strength of the actuators, which were constant. Thus, the results suggest that an alternate explanation of why the muscles of the upper leg are larger is that it may create a mass distribution which enables improved physical dynamics for locomotion. Controllability The fact that all three experimental conditions yield very similar final morphologies, despite a fairly large range of possible morphologies, also lends credence to another hypothesis: that some morphologies are easier to control than others for a given task. Intuitively, it can be seen why this is the case. Consider, for example, two bipeds with identical leg structures, one with large flat feet and the other point contacts. It is clear that the biped with flat feet is easier to stabilize during walking than the one with point contacts. Generalizing from this example, we can see that the shape of the body influences the ease of control, and thus, in effect, the overall neural control requirements. Intersegmental Dynamics and Neural Control In addition to the shape of the body influencing the overall neural requirements, the morphology also affects the specific design of the neural control architecture, through the intersegmental dynamics. To demonstrate this concept, consider a three link kinematic chain, connected by two rotary joints, as shown in Fig. 9-1. The top link is connected to the middle link by a rotary joint with an actuator, which applies a torque % . The middle link is connected to the bottom link by another rotary joint, labelled P, which is able to move passively or actively. (The bottom link is considered to have insignificant mass.) If mass &(' is much greater than &*) , in the absence of other external forces the movement of the middle link will be larger than that of the top link. This will cause passive movement in joint P, which will cause the bottom link to move to a greater extent. Conversely, if mass 126
m1
m1
τ
τ
m2
m2 P
P
Figure 9-1: Effect of mass distribution on neural control through the intersegmental dynamics
+*, is much larger than +(- , the top link will move much more than the middle link, and thus the passive movement at joint P will be much smaller. Thus, in a task where the movement of joint P is useful, the second situation will require more active control of joint P than the first. This demonstrates the mechanism by which simply changing the mass distribution can redistribute the amount of neural control required at various parts of the body. Morphology and local sensory information The body also affects the neural control architecture from the point of view of sensing. Neural pathways which convey sensory information are “expensive” so to speak, for two reasons. One is that long neural connections have to be made between separate neural modules to convey the afferent information. Secondly, the more sensory inputs a neural module must integrate, the more complicated the resulting neural architecture becomes. Thus, from the point of view of economy of design it makes sense for a neural architecture to be more modular, and reduce the number of connections to distal sensors. The results of Chapter 3 show that such an architecture can be supported by using the dynamics of the body for communication between disconnected neural modules. The results show that in a biped structure in which the joints are compliantly controlled, contralateral information can be communicated via the dynamics of the body. The reason this is possible is that the results of the movement of one leg affect the attitude of the waist, which is the connecting link between the two sides. As the waist (and upper body if one exists) is usually massive, small changes in the attitude of the waist influence the position of the center of gravity of the body, and as a result the total force supported by the contralateral leg. Unless, the joints of this leg are completely rigid, this change in force will lead to a change in the joint angles, which can be measured by the local sensors. This can be exploited by the neural architecture to effectively coordinate disconnected neural modules. In the case of contralateral coordination, this means of communication is more effective in stance phase rather than in swing phase, due to the fact that the ground constraint forces the joints of the leg to react to the change in supported weight. When the leg is in swing
127
phase, the change in the attitude of the waist may not have as great an influence on the movement of the leg. Nonetheless, the fact that such communication is possible points to a more general principle which may underlie the neural control of movement, i.e. that local sensors can convey remote sensory information1. Morphology and global sensory information A second way in which the body can be used to coordinate disconnected neural modules is through global sensing. The results of Chapter 3 show that in a biped morphology, certain global sensors can be used to coordinate disconnected neural modules. One reason this is possible is that in the anthroform biped morphology, the attitude of the upper body is quite sensitive to changes in the support structure provided by the legs. If a leg changes from being in contact with the ground, and supporting the weight of the body, to being off the ground, the upper body attitude is affected due to the physical dynamics. Thus, global sensory information related to body attitude indirectly contains information about the movement of the other leg. A neural architecture can then use this information to reduce the requirements on the neural substrate needed for the coordination of disconnected modules. Morphology and Computation The results of Chapter 4 theoretically demonstrate that an intrinsic interaction can exist between the body and its control from a computational perspective. The dynamics of the physical structure of the body determines the transformation from the motor commands to sensory inputs for the controller. As changing the physical shape of the body can change the computation performed by it, it can in turn greatly influence the computational requirements of the controller. This is may be one underlying cause of the morphology and control trade-off.
9.3
Material Properties
Chapters: 4, 5, 6 Although not directly addressed, Chapter 4 clarifies to some extent how material properties can enter into the relationship between the body and neural control. Properties such as joint stiffness, compliance of contact surfaces and mechanical elasticity can drastically change the mapping between actuation and its physical consequences, which can have a great effect on control requirements. Chapters 5 and 6 both demonstrate this effect. In the STUMPY robot described in Chapter 5, mechanical compliance is included at the feet via compression springs. These springs have the effect of allowing energy from the motion of the upper body to be stored in the lower body, and to be released into the lift-off phase, enhancing hopping. In the absence of this compliance, the lift-off phase would be greatly reduced in duration and amplitude, most likely requiring additional upper body actuation to compensate. In 1
This principle was also shown to be used in the neural control of insect locomotion by Cruse et al. [17]
128
the BENDY robot, described in Chapter 6, this effect is even more clearly demonstrated. The spring included between the torso and shoulders serves to amplify the effect of the rather weak torso actuator. In the absence of this the rotational inertia generated due to torso actuation at the lower body would be insufficient to generate lift-off, which would call for additional control either in the joints of the upper or lower body. Thus, the material properties of the morphology can have a significant effect on the control requirements.
9.4
Role of the Upper Body
Chapters: 5, 6 Traditionally, the role of the upper body has been viewed as a mass to carried along, or as a mass to be balanced as in an inverted pendulum, or a means of keeping balance by reacting to changes in body attitude with postural adjustments. In all cases, the common feature has been that the upper body has been thought to have a static or quasi-static role in the dynamics of gait generation. Thus, neural architectures for walking are mostly designed to generate gait using the joints of the lower leg, and control posture using the joints of the lower leg and upper body. In our work we have shown that this is an abstraction made by human designers, which may not be necessary or even correct in biology. In the work with the STUMPY robot, a robot which models the waist and shoulder joint of the upper body, and only has short passively compliant “stumps” for legs, the potential contribution of the upper body to the dynamics of gait generation are highlighted. It is shown that the use of the rotary joint at the waist for side-to-side oscillation can not only support lift-off by transferring the weight to the other side of the body, but in fact generate lift-off through the dynamics of the body. The rotation of the upper body in the frontal plane will generate rotational momentum in the entire body, which will cause the leg to lift off the ground. Similarly, it is shown that the rotation of the shoulder in the transverse plane can actually generate forward motion of the swing leg. The motion of the shoulder generates rotational momentum around the vertical axis, which translates down through the body causing the swing leg to move. The motions of STUMPY are highly exaggerated and not human. This is because unlike a human, it can only use its upper body for locomotion. Thus, it must exert much greater torques through its upper body, which looks very unnatural. The aim of the work with STUMPY was not to suggest that in human locomotion, the upper body is solely responsible for gait generation. Rather, the aim was to highlight the potential contribution of the upper body to the dynamics of bipedal gait generation. The work with STUMPY was insightful and generated new hypothesis for biped locomotion. However, the fact was, it still only had stumps. Although it could be hypothesized that the dynamics of the upper body could be used for biped locomotion, there was still no physical proof. STUMPY barely qualified as a biped2 . Thus, a real physical demonstration of the contribution of the upper body in biped locomotion was lacking. 2 Perhaps it could qualify as a quadruped, but can stumps with no rotational degrees of freedom be called legs at all?
129
The BENDY robot was developed to fill this void. It was a real biped robot with three rotational degrees-of-freedom in the sagittal plane in each leg, corresponding to ankle, knee and hip joints. The ankle and knee joints were passively compliant, and the hip joints were passive with a small range of motion. The focus was on the contribution of the upper body motion in the frontal plane for gait generation, and thus the robot only had one actuator in the entire body, which was at the waist. Using this actuator for side-to-side oscillation, it was shown that as predicted, lift-off could in fact be generated. Furthermore, it was found that if at the time of lift-off the position of the foot was posterior to the waist in the sagittal plane, the dynamics would serve to generate a passive forward swing in the absence of additional control. The robot was a proof that the upper body motion could induce dynamics which would assist in gait generation. Once again, it must be noted that the BENDY robot is an abstraction from the biological situation, in order to prove a point. It should not be understood from this work, that the majority of the responsibility for gait generation lies in the upper body. From EMG data of human locomotion, it can be clearly seen that the muscles of the lower body are highly active during gait generation [119]. However, the goal is to demonstrate that the division between the roles of the upper and lower body in human locomotion may not be as clear cut as was previously understood. This insight has implications for further understanding the neural control of human locomotion. Previously it has been considered sufficient to analyse and model the spinal cord and its control of the lower legs as the main seat of gait generation. However, in light of these results, it is likely that such an approach will yeild an incomplete or even incorrect view, as it will ignore the effects of the inherent dynamics of the human morphology in neural control.
9.5
Proprioceptive vs. foot contact information
Chapters: 7, 8 There are many sensory modalities which have been described with respect to walking: joint angle information such as the anterior extreme position (AEP) and posterior extreme position (PEP), muscle length information, muscle force information, and foot contact information. Some of these have redundant functionality. For example, the PEP is used to signal the switch between stance and swing phase and AEP is used to signal the switch from swing to stance, which is similar to the function of the cutaneous reflexes. In the human system, it is not known whether all of these reflexes exist, if they are involved in walking, and which are more significant than others. As far as the AEP and PEP, the function of these reflexes have been described as fairly redundant with that of the cutaneous reflexes. Moreover, the results from Chapter 7 suggest that although the cutaneous reflexes can support rhythmic stepping in the absence of joint angle information, the reverse is not true. That is, rhythmic stepping in a biped is not easy to generate using joint angle sensing alone. Thus, it suggests that cutaneous reflexes are much more likely to be used in the regulation of swing and stance phase than joint angle sensing. This of course, does not suggest that these reflexes may not exist. Biological 130
systems are extremely redundant in nature, so it may be that joint angle based reflexes exist which serve a secondary role to the cutaneous reflexes during walking. The results of both Chapter 6 and 7 suggest that the muscle spindle and golgi tendon organ related reflexes are also less important than the cutaneous reflexes. In Chapter 6 this is demonstrated by the fact that the gain of the proprioceptive reflexes is lower than that of the cutaneous reflexes in a healthy walking condition. Also, in Chapter 7 it is shown that the biped can walk without receiving any information related to joint torque (which would be related to muscle force) or joint angles or angular velocity (which would be related to muscle length and velocity). Thus, it can be hypothesized that the proprioceptive reflexes also play a secondary role in walking compared to the cutaneous pathways.
9.6
New model of spinal neural architecture
Chapter: 7 Many of the previous neural spinal cord models have suggested that the neural system has internal states which relate to the “phase” of the system. In the case of Taga, he postulated six internal states which could be used to regulate the activation of the CPGs. In the model by Wadden and Ekeberg, they use a neural pattern generator which has four neural modules, one corresponding to each phase of walking. The switching of activation between the phases helps to modulate the effect of the reflexes on the system. However, our system is unaware of phases. We have shown that by simply using central pattern generators which provide rhythmic output and reflexes working in parallel, walking can be produced. Furthermore, many other models include direct neural connections between the central pattern generators and the reflexes. Our model shows that such connections are not necessary, and that by working as loosely coupled parallel processes the sub-systems can coordinate through the dynamics of the body, in the interaction with the environment, to produce walking. The question of whether the central pattern generator has sensory inputs is not a question that we have attempted to answer using this model. In all likelihood, the central pattern generator does have sensory inputs as in Taga’s model, which are used for entrainment purposes. This leads to greater stability of the system against disturbances, while allowing for a large range of variability in the speed, as shown by Taga. However, our work shows that the only neural structures necessary for generating walking, in a biped with an upper body, are central pattern generators and reflex pathways in parallel. If these structures exist it is not necessary for the system to explicitly represent the phase of walking. In effect, it is a much simpler model which explains the behavior of human locomotion.
9.7
Coordination of CPG and Reflexes
Chapters: 3, 5, 6, 8
131
In the spinal cord model as it was used in Chapter 6, there was no problem with coordinating the CPG and the reflexes because there was no change in desired speed during walking. But if the CPG would like to command a change in speed then it is possible for a conflict to arise between the commands of the CPG and the system of regulation of rhythm afforded by the reflexes. This problem exists whether or not the CPG has sensory inputs. For example, consider that a leg is in stance phase, and the CPG commands it to change to swing phase. Now since the other foot is still in the air it cannot lift off the ground without falling. Thus, the cutaneous reflex commands it to remain in swing phase raising a conflict between the CPG and reflex commands. One solution may be that the effect of the CPG on activating commands for swing phase are weak, and can therefore be easily overridden by the commands from the reflex. However, this would still imply that the system is fighting itself, which would not be efficient. The results of Chapters 3-6 suggest an alternative solution of how this could function using the dynamics of the body. In Chapters 5 and 6, it was shown that the upper body motions could regulate the motions of the lower body, specifically the lift-off of the feet. Since the lift-off of the feet effects the cutaneous sensors, and Chapter 8 demonstrated that cutaneous reflexes were sufficient for the generation of gait, it indicates that oscillation of the upper body in the frontal plane would be sufficient to alternate the activation of the right and left cutaneous reflexes such that gait would be produced. Furthermore, the frequency of upper body oscillation would directly correlate to the alternating activation of the cutaneous sensors, so an increase in oscillation frequency would directly result in faster stepping. Now there are at least two possible ways in which the CPG can coordinate with this system to increase walking speed. One is to override the effect of the reflexes directly and control all the joints of the lower leg to move faster. This will lead to a transient phase in which the reflex commands will not be synchronized with the CPG followed by an entrainment phase where the CPG and reflexes will once again synchronize to reach a new equilibrium. The second possibility, however, is to use the mechanism described in Chapter 3 of exploiting the dynamics of the body to coordinate the disconnected neural modules. Using this approach the CPG could coordinate with the reflexes by simply commanding a change in the frequency of the upper body and letting the physical dynamics communicate the message to the reflexes. [This hypothesis was presented briefly at the end of Chapter 8]. It can be seen that this hypothesis has several advantages over the other models. Firstly, it means that much fewer neural oscillators are required for locomotion as it is not necessary for every joint of the lower leg to be controlled by a neural oscillator. Secondly, it means there are fewer neural connections to each motoneuron, as the CPG does not need to connect to each of the joints of the lower leg. Thirdly, by eliminating the need for neural oscillators at every joint, it also eliminates the need for explicit communication between these joints, which would have to be accomplished by more neural pathways. Finally, it does not give rise to an occasion when different neural modules send conflicting commands to the same joint. In summary, by including the dynamics of the body, it gives a much simpler explanation of the phenomena than other models.
132
9.8
Products
In research which is purely theoretical the main contributions are usually scientific. But in research which is both theoretical as well as applied, the contributions can be both scientific as well as practical. The following sub-sections will summarize the practical contributions of the research in terms of the software tools and robots which have been developed.
9.8.1
Neuro-musculo-skeletal model
The Neuro-Musculo-Skeletal model was introduced in Chapter 7, and it was shown how it could be used to investigate scientific issues regarding the role of neural structures of the spinal cord such as the central pattern generators and reflex pathways. The components of the model were described, and the methods which used for validating the model were outlined. The scientific shortcomings of the model were then discussed and ideas were presented for how they could be overcome in future work. In addition to providing a method of scientific investigation, a main goal of the model is to serve as a tool for physicians to investigate the effects of spinal lesions on locomotor abilities, in order to compare them to the observed pathologies of their patients. Therefore in addition to the scientific considerations, it is also necessary to consider the usability of the tool as a software package. Currently, the software, which runs under Windows 2000, has two main components: a graphical neural network design software, NNetview3 , which is used to display and enable changes in the spinal neural architecture, and a biomechanical human body model running in Matlab. The use of the tool by physicians proceeds as follows. They first effect a “spinal lesion” by deleting part of the spinal neural architecture in NNetview (Fig. 9-2(a)). Then they export the topology of the network into a text based representation (Fig. 9-2(b)). Then they start Matlab and run the biomechanical body model program which loads the text file describing the network. It builds a neural network based on it, and uses it as a controller for the body whose motions are calculated every 0.0001 second using standard rigid-body dynamical equations of motion (Fig. 9-2(c)). When the simulation is complete all the data is stored in arrays (Fig. 9-2(d)), and they can use the “Display Plots” function to choose between one of 19 options for visualizing graphs related to the motion of the body. They can also replay the movie of the simulation. The four stages of the process can be summarized as follows: 1. 2. 3. 4.
Lesion spinal architecture using NNetview Export as text file Run biomechanical body simulation in Matlab Display data / Replay movie
This process could be improved in certain ways. At the moment, there are some steps that have to be performed manually which could be eliminated. For example, after a lesion is made, the network has to be manually exported as a text file and then the Matlab program 3
Developed by Neuronics Inc, a spin-off company of the AI Lab, University of Zurich
133
Figure 9-2: NSCM software tool for clinical studies: (a) NNetview graphical neural network design tool (b) network represented as text file (c) Biomechanical body model controlled by neural network, running under Matlab (d) Movement data stored in Matlab arrays has to be run. It would be more efficient if the physician would only be required to make a lesion and choose the types of graphs that he or she would like to have displayed, and have the rest proceed automatically. Another issue, is that the performance of the tool is rather slow. It simulates 1 seconds of walking data in about 2 minutes. This is because high accuracy integration methods are required and it is running in Matlab. The future goal of the software implementation will be to steer away from the use of proprietary software, in particular Matlab, and reimplement the system in C++. This will make the system faster as well as more streamlined in design. Another goal from the perspective of the user interface, will be to design an integrated GUI which visualizes the neural network, the biped movie, and the data plots, in the same window. The GUI will allow the physician to select certain parts of the network to lesion, indicate using checkboxes the graphs he or she would like to visualize and then press “Run”. The program should then run the simulation, store the data, and graph the desired data automatically.
134
9.8.2
STUMPY
The STUMPY robot, whose control was studied in the course of this work, is a hopping robot platform which requires minimal control for locomotion. It is one of the first nonwheeled robot platforms which can control both position and orientation using only two actuators. The two actuators are controlled using simple joint angle based sensorimotor reflexes to produce a variety of gaits including shuffling (feet never leave the ground), walking (only one foot is off the ground at a time) and hopping (both feet are off the ground in certain phases). Using these gaits the robot can move forwards, reverse and turn at various turning rates. The practical contribution of STUMPY is considerable as a locomotion platform for various kinds of terrain. It has been shown by Iida [47], that using the control strategies developed, the robot can move both on surfaces with high friction and low friction. Furthermore, the self-stabilizing dynamics achieved as a result of interaction of the morphology with the control enable the robot to traverse mildly uneven surfaces and overcome minor obstacles without additional control. The robot is also able to recover from moderate external perturbation caused due to accidental collisions or human intervention. It also does not require additional computation for the control of starting and stopping. Augmented with Figure 9-3: STUMPY visual sensing, the robot will serve as a versatile navigation platform.
9.8.3
BENDY
The BENDY robot, which was developed in the course of this work, is one of the first biped robots to walk using passive legs on level ground and using only a single actuator. The practical contribution of the BENDY robot is both small and very large from different perspectives. From the perspective of the development of an effective humanoid robot platform to carry out tasks in the office or home environment, BENDY is a first prototype and as a result much too impractical. With completely statically stable walking at the small speed of 2 cm/sec BENDY would not even be a practical choice to carry a letter from one office to the next. Moreover, with the wide range of motion of the upFigure 9-4: BENDY per body, BENDY represents more of an environmental hazard when it moves, than a friendly office aid. Yet the implications of the BENDY robot for future humanoid robot locomotion are very far reaching. In the future, humanoid robots will be more involved in supporting day to day activities in home, office and factory environments. It will be of utmost importance that these complex robots have cheap, robust, and energy efficient means of locomotion as found in natural systems [1]. The BENDY robot leads the way to this goal. By demonstrating how the upper body can be used to create passive motion in the lower body, the 135
robot illustrates how the actuation of humanoids can be drastically reduced for locomotion. Furthermore, it illustrates how the upper body motion can be used to create “safe” windows of opportunity for lower leg movement from the perspective of balance, so that a simple sensory motor coupling between the upper body proprioceptive sensors and lower leg movement could be used to support the basic function of locomotion on level ground. Thus, the computational requirements of basic walking could be greatly reduced, leaving more computational resources for higher level adaptive behaviors. A good example of this approach can be seen in the development of the VIKI robots [64], based on the work of Paramonov and Lund [77], which won the RoboCup Humanoids Free Style Championships in 2002.
9.9
Personal Experience
This section is a short digression from the scientific discussion of locomotion, to the short description of a personal experience. Often it can be of great use when personal life coincides with research, giving the researcher both a view from the inside and the outside of the system, so to speak. So although the experience that will be described is an unfortunate one, I hope that it will have something to add from the perspective of locomotion research. Towards the end of my PhD work, in the January of 2003, I suffered a moderately serious head injury. The accident occurred while I was walking down some concrete steps in Irchel park, which had become icy due to a particularly long stretch of sub-zero temperatures. I slipped on one of the steps and my body accelerated in such a way that the back of my head hit the step above, very hard. For a few minutes, I couldnt get up but when I did I seemed to be able to walk and otherwise function normally so I did not make much of it. I continued with my normal activities. It was only the next day that I realized that something was wrong. Although, I tried to get out of bed, I could barely get up. It seemed that there was no force in my body to move. Finally, I convinced myself that forcing myself to get up would be the right thing to do. It was only then that the full impact of what was wrong hit me. I could no longer feel the back of my head. On the right side of my head, I had a searing ache above my ear which got worse when I tried to think or feel anything. Finally, when I tried to walk I realized that I could no longer balance. I could only wobble around as if I was drunk. The first days were quite terrible, since I could neither move effectively nor think or feel without activating the ache at the right side of my head. But after a while, when the thinking got a bit easier, I started to take more of an interest in the fact that I could not walk properly. I started to run tests on myself. First I wanted to understand if the balance problem was a general one, or whether it had affected one side more than the other. I tried to balance on one leg at a time. I found that although I could balance perfectly on my right leg, when I tried to balance on my left leg I fell over immediately. Furthermore, I found that I could also not “feel” my left foot. For example, when I stood on my right foot I could feel the weight of my body on my foot, and whether it was closer to my heel or my toes. But with the left foot I experienced a kind of numbness that did not change whether I leaned forwards or backwards. Also, touching the sole of my right foot caused much more 136
sensation than touching the sole of my left foot. I had a medical examination including CAT and MRI scans which revealed that there was no physical damage in my brain. However, the doctors tests revealed that there were measurable functional problems. For example, the tendon tap reflex on my left knee produced much less of a response than that of my right knee. Also, I could feel a small vibration applied to the tendon of the right M. Soleus muscle, but for the left one the vibration amplitude had to be increased four times before I could experience any sensation. When I walked, there was also an observable discrepancy in my gait. When I stepped on my right leg, things went fine, but when I stepped on my left leg, the knee started to buckle in mid-stance. In our research we had based our work on the premise that the spinal cord was the main seat of locomotion, and that as long as the central pattern generators and reflex pathways were functional, locomotion could be recovered. But here I was with mainly a right brain injury, and nothing particularly wrong with my spinal cord, not able to walk properly. Furthermore, I was not even able to produce appropriate reflex responses while lying down. The reality of this problem woke me up to the fact that the interdependency between the supra-spinal centers and the reflexes was much greater than we had previously imagined. From an engineering perspective, it would have been nice if the reflexes were independent spinal sensory-motor processes which mostly worked irrespective of central control, or if the central control only played a marginal role in regulating the gain at key points in the gait cycle. But from my experience, it seemed that this was not the case. It seemed that the whole reason the spinal reflexes were active was because they were commanded to be active during certain phases of the gait cycle by the supra-spinal centers. And in my case, since those supra-spinal centers on the right side were not functioning appropriately, the reflexes on the left side mostly remained inactive. Furthermore, it seemed that both proprioceptive and cutaneous reflexes were regulated by supraspinal control. In my case, the fact that the proprioceptive reflexes were affected was clear: the tendon tap reflex result was very poor. Also, from the work described in Chap 7, I knew that the buckling of the knee in mid-stance can be caused by two factors: either the stretch reflex gains are too high or the cutaneous reflex gain is too low. This meant that since my proprioceptive reflex gains were obviously not too high (as shown by the results of the tendon tap test at the doctors’ office) the most likely reason my knees were buckling during mid-stance was that the cutaneous reflex gains had also become too low. This correlated well with the fact that I could also no longer “feel” my cutaneous sensors. Reflex gain modulation is an area which has not been well understood. Although, it has been known that it is an integral part of locomotion, as measured in experiments, it has not been known whether such gain modulation is spinal or supraspinal. Some spinal cord models such as ours, leave out reflex gain modulation with the assumption that it is a supraspinal function. Others, such as Ekeberg, Rybak, and Hase include interconnections between the CPG and the reflexes, so that reflex gain modulation can occur at a spinal level. The experiences described above, however, suggest that reflex gain modulation is most likely a supraspinal function. Although this means that technically our model is more correct in describing the spinal architecture, the fact that it implements walking in the absence of supraspinal gain modulation may be attributing too much functionality to the spinal cord. Future work should focus on investigating and integrating the supraspinal 137
modulation of spinal reflex pathways into the model, as without this our understanding of locomotion is likely to remain incomplete.
9.10
Implications for Rehabilitation after Spinal Injury
The premise we started out with was that rehabilitation of the spinal reflexes below the level of a spinal injury would be beneficial to the recovery of locomotor function in a patient. Our work bears evidence to this fact. The pathological function of the reflexes, i.e. cutaneous reflexes being too low, or stretch reflexes being too high can both lead to disruptions in the locomotor pattern. Thus, returning the reflexes to their appropriate functional contributions is essential. Furthermore, it is likely that Central Pattern Generators are stimulated by afferent inputs, and externally moving the legs so that these centers are stimulated by the afferent inputs is most likely very beneficial for spinal recovery. Body Dynamics However, our results suggest that it is extremely important to enable the body to experience its own natural dynamics during therapy. It is extremely important that the movements of the upper body and arms can translate to the lower body, and that the links can experience intersegmental dynamics, in order for the neural system to receive the correct pattern of inputs. Currently, therapy using the Lokomat driven gait orthosis falls short of this requirement. The Lokomat constrains the hip movement of the patient in all directions except the vertical axis, in effect, decoupling the dynamics of the upper and lower body. It also moves the patients legs through a fairly rigid pattern of motion, which minimizes the effect of intersegmental dynamics. One way to improve this would be to include more passive degrees-of-freedom in the Lokomat to enable free translation and rotation of the hip in the frontal plane, as well as rotation of the hip in the transverse plane. This would enable the dynamics of the upper body to translate to the lower body. Another approach would be to detach the exoskeleton from the base of the treadmill so the body would be free to move within the constraints of the suspension harness. The problem could also be more effectively addressed through a redesign of the Lokomat, where the electromotors were replaced with pneumatic actuators, enabling both active and passive movement in the exoskeleton and increasing the possibility for intersegmental dynamics. For therapy purposes, in addition to therapy using the Lokomat, it may be beneficial to consider the use of Functional Electrical Stimulation (FES) of the patient’s muscles to generate the gait pattern4 , as this method preserves the intersegmental dynamics of the system. Complete system perspective Furthermore the idea of simply studying the spinal cord to understand spinal cord injury is not sufficient. When the spinal cord is injured, not only are spinal interneurons which mediate reflexes and rhythmic activity impaired, but also the regulation of these reflexes from the higher brain centers. If the connections from the brain stem to the spinal cord 4
in a situation where the patient is freely suspended over a treadmill
138
are impaired for example, then a patient will experience difficulty in the control of posture, which is a prerequisite for walking, i.e. if you cannot stand up, then you cannot walk. Thus, future studies of spinal cord injury must take a more complete view of the system. Adaptive Organization It is true that it must be understood how the neural system works, but simply finding out how it works in a particular instance is not enough to understand the adaptivity which sustains it as a structure. It still does not help us answer the question “Why or how does the spinal cord reorganize itself after a spinal cord injury?” and “To what extent is spinal reorganization possible?” Is it because there are extra spinal neurons which can become useful in the case of an injury? Or is it that neurons which were previously responsible for other less important functions get recruited for more important functions? In order to understand how the spinal cord recovers, it is necessary to study the process of recovery itself more closely. For example, studying the factors which lead to the recovery of a certain previously unavailable reflex can give us clues as to whether the reflex was inactive because 1) the pathways activating it are severed 2) the pathways inhibiting it have become overactive 3) the interneuron mediating the reflex itself has become damaged. In either case, the process which enables the system to reconnect or reroute the necessary information is the most important from the perspective of rehabilitating spinal cord injury. Thus, the main insight is that the neural architecture itself is a product of the processes which help to maintain it as a structure. Therefore, simply studying the architecture itself will not get to the root of the problem. The root of the problem is to understand the processes which help to organize the structure. This should be a more prominent focus of future work. Categorizing Injuries There are two kinds of neuronal impairments that need to be distinguished, functional vs. physical. A functional injury is something that is suffured for example in a concussion or a whiplash situation, where neural activities can become altered due to extreme physical stimulation, without neural death. The second kind of problem is physical when the cells themselves become destroyed or die. By identifying and focusing on these two separate categories of problems, further progress can be made. Axonal Regeneration During the course of this work, there has been some progress in the field of axonal regeneration. Although still a topic of research, recent results [90] show that it may be possible to induce some axonal regeneration with the right pharmacological treatment, leading to recovery of function. Future work should investigate the possibility of combining axonal regeneration with physical therapy techniques.
139
9.11
Summary of Contributions
In summary, the work presented in this thesis makes contributions to three domains. The primary contribution is to the understanding of human locomotion. By illuminating how the dynamics of the biped morphology in interaction with the environment influence neural control, it sheds light on a previously unexplored area of human locomotion. The secondary contributions of the work are to robotics and engineering. Through the development of robotic platforms which exploit the dynamics of the morphology to reduce control, it delivers a versatile navigation platform and demonstrates how the design and control of humanoid robots can be simplified. Finally, it makes contributions to clinical practice. The new insights into human locomotor control lead to recommendations for improvements in therapy techniques. Furthermore, the development of a software package which enables a patient’s condition to be mapped to an internal pattern of neural lesions provides the physician with an invaluable diagnostic tool for the treatment of spinal cord injury.
140
Bibliography [1] Alexander, M. (1990) Three Uses for Springs in Legged Locomotion. In Int. J. Robotics Research, 9:2, p. 53-61 [2] Aubert X (1956) Le couplage de la contraction musculaire. Edition Arscia Brussels, Thesis [3] Bongard, J. C., and Paul, C. (2000) Investigating morphological symmetry and locomotive efficiency using virtual embodied evolution. In The Sixth International Conference on the Simulation of Adaptive Behaviour, Paris, France, p. 420-429. [4] Bongard, J. and Paul, C. (2001). Making Evolution an Offer It Can’t Refuse: Morphology and the Extradimensional Bypass, in Proc. Sixth European Conference on Artificial Life , Prague, CZ. [5] Boone G. N., and Hodgins, J. K. (1995) Reflexive Responses to Slipping in Bipedal Running Robots, IEEE Int. Conf. Intelligent Robots and Systems, p. 158-164. [6] Braitenberg, V. (1984) Vehicles: Experiments in Synthetic Psychology. MIT Press Cambrige, MA [7] Brooks, R.A (1989) A Robot that Walks; Emergent Behaviors from a Carefully Evolved Network, Neural Computation, 1(2):253-262 [8] Brooks, R.A (1991) Intelligence without Representation. Artificial Intelligence 47, 139-160 [9] Brooks, R. A., and Stein, L. A. (1994) Building brains for bodies. Autonomous Robots, 1(1):7–25. [10] Brown, T. G. (1914) On the nature of the fundamental activity of the nervous centers; together with an analysis of the conditioning of rhythmic activity in progression, and a theory of the evolution of function in the nervous system. Journal of Physiology, London, Vol. 48, p. 18-46. [11] Buehler, M., Battaglia, R., Cocosco, A., Hawker, G., Sarkis, J. and Yamazaki, K. (1998) Scout: A simple quadruped that walks, climbs and runs In: Proc. Int. Conf on Robotics and Automation, p. 1707-1712.
141
[12] Calancie, B., Needham-Shropshire, B., Jacobs, P., Willer, K., Zych, G., Green, B. A. (1994) Involuntary stepping after chronic spinal cord injury. Evidence for a central rhythm generator for locomotion in man. Brain, 117(5):1143-59 [13] Chocron, O. and Bidaud, P. (1999) Evolving walking robots for global task based design. In Proceedings of the Congress on Evolutionary Computation, Washington, USA, p. 405-412. [14] Chou, C. P., Hannaford, H. (1997) Study of human forearm posture maintenance with a physiologically based robotic arm and spinal level neural controller. Biological Cybernetics, Vol. 76, p. 285-298. [15] Collins, S. H., Wisse, M., and Ruina, A. (2001) A 3-D passive-dynamic walking robot with two legs and knees. In International Journal of Robotics Research, [16] Colombo G, Joerg M, Schreier R, Dietz V (2001) Treadmill training of paraplegic patients with a robotic orthosis. J Rehabil Res Dev, 37:693-700 [17] Cruse, H., Bartling, C., Dean, J., Kindermann, T., Schmitz, J., Scumm, M. and Wagner H. (1996) Coordination in a six-legged walking system: Simple solutions to complex problems by exploitation of physical properties. In Proc. 4th Int. Conf. on Simulation of Adaptive Behavior, Cape Cod, USA, 84-93 [18] Degtyarenko, A. M., Simon, E. S., Norden-Krichmar, T. and Burke, R. E. (1998). Modulation of oligosynaptic cutaneous and muscle afferent reflex pathways during fictive locomotion and scratching in the cat. J Neurophysiology, Jan 79(1), p. 447-63. [19]
Delcomyn, F. (1980) Neural basis of rhythmic behavior in animals. Science, 210(4469), p. 492-498.
[20] Dietz, V., Colombo, G. and Jensen, L. (1994) Locomotor activity in spinal man. Lancet, 344:1260 [21] Dietz, V. (2002) Do human bipeds use quadrupedal coordination? Trends in Neurosciences 25(9):462-467 [22] Dietz, V., Mueller, R., Colombo, G. (2002) Locomotor activity in spinal man: significance of afferent input from joint and load receptors. Brain, 125:2626-2634 [23] Duysens, J. (1977). Reflex control of locomotion as revealed by stimulation of cutaneous afferents in spontaneously walking premammillary cats. J Neurophysioly, Jul;40(4):737-51 [24] Endo, K., Yamasaki, F., Maeno, T., and Kitano, H. (2002). Optimal Biped Walking for Humanoid Robot by Co-evolving Morphology and Controller, In PRICAI, 325334
142
[25] Endo, K., Maeno T., and Kitano, H. (2002). Co-evolution of Morphology and Walking Pattern of Biped Humanoid Robot using Evolutionary ComputationConsideration of characteristic of the servomotors, In Proc. Int. Conf. Intelligent Robots and Systems, pp 2678-2683 [26] Espiau, B., and Sardain, P. (2000) The anthropomorphic biped robot BIP2000. Proceedings of the 2000 IEEE Conference on Robotics and Automation, 4:3996–4001. [27] Fukuda, T., Komota, Y., and Arakawa, T. (1997) Stabilization control of biped locomotion robot based learning with GAs having self-adaptive mutation and recurrent neural networks. In Proc. IEEE Int. Conf. on Robotics and Automation pages 217222 [28] Furuta, T., Okomura, Y., and Tomiyama, K. (2000) Design and Construction of a Series of Compact Humanoid Robots and Development of Biped Walk Control Strategies’, CD-ROM Proc. IEEE/RAS Conference on Humanoid Robots, 84.pdf [29] Gallagher, J. C., Beer, R. D., Espenschied, K. S., and Quinn, R. D. (1996) Application of evolved locomotion controllers to a hexapod robot. Robotics and Autonomous Systems, 19:95–103. [30] Garcia, M., Chatterjee, A., Ruina, A., Coleman, M. (1997) The Simplest Walking Model: Stability, Complexity and Scaling. ASME Journal of Biomechanical Engineering. [31] Geyer H., Seyfarth A., Blickhan R.(2003) Positive force feedback in bouncing gaits? Proc R Soc Lond B Biol Sci, Vol. 270(1529), p. 2173-2183. [32] Goswami, A., Thuilot, B., and Espiau, B. (1998) A study of the passive gait of a compass-like biped robot: symmetry and chaos . International Journal of Robotics Research, Vol 17, No. 12, 1998. [33] Grillner, S. (1985) Neurological Bases of Rhythmic Motor Acts in Vertebrates Science, 228:143 ¨ Manira, A. El, Hill, R. H., Lansner, A., [34] Grillner, S., Deliagina, T., Ekeberg, O., Orlovsky, G.N., and Wall´en, P. (1995) Neural networks that co-ordinate locomotion and body orientation in lamprey. Trends in Neuroscience 18(6): 270-279. [35] Grillner, S. and Zangger, P. (1979) On the central generation of locomotion in the low spinal cat. Experimental Brain Research 34:241-261. [36] Guenther M (1997) Computersimulationen zur Synthetisierung des muskulaer erzeugten menschlichen Gehens unter Verwendung eines biomechanischen Mehrkoerpermodells, Dissertation of the University of Tubingen [37] Hase, K. and Yamazaki, N. (1999). Coevolution of human bipedal walking by a neuro-musculo-skeletal model. In Artificial Life and Robotics, 3, pages 133-138
143
[38] Hatze, H. (2001) An efficient simulation method for discrete-value controlled large scale neuromyoskeletal system models. Journal of Biomechanics, 34:267-271 [39] Hayashi, R. and Tsujio, S. (2001) High Performance Jumping Movements by Pendulum-type Jumping Machines In Proc. IEEE/RSJ Int. conf. on Intelligent Robots and Systems, Maui, USA. p. 722-727. [40] Haykin, S. (1994) Neural Networks: A comprehensive foundation. Macmillan College Publishing Company, New York [41] Van Heijst, J. J. and Vos, J. E. (1997) Self-organizing effects of spontaneous neural activity on the development of spinal locomotor circuits in vertebrates. Biological Cybernetics, 77:185-195 [42] Van Heijst, J. J., Vos, J. E. and Bullock, D. (1998) Development in a biologically inspired spinal neural network for movement control Neural Networks, 11:13051316 [43] Hemami, H. and Farnsworth, R. L. (1977) Posture and gait stability of a planar five link biped by simulation, IEEE trans. on Automatic Control, AC-22. [44] Hill AV (1938) The heat of shortening and the dynamic constants of Muscle Proc. Roy. Soc., 126(B):136-195 [45] Hirai, K., Hirose, M., Haikawa, Y., and Takenaka, T. (1998) The development of Honda humanoid robot In IEEE Proc. Conference on Robotics and Automation. [46] Hirose, M., Haikawa, Y., Takenaka, T., and Hirai, K. (2001) Development of Humanoid Robot ASIMO Proc. Int. Conference on Intelligent Robots and Systems. [47] Iida, F. Exploiting Friction for the Locomotion of a Hopping Robot, inProc. Int. Symp. Adaptive Motion of Animals and Machines Kyoto, Japan [48] Ioi, K., Igarashi, H. and Muramami, A. (2001) Design of a gravitational wheeled type robot In Proc. International Symposium on Robotics, Seoul Korea, p. 175-179. [49] Ijspeert, A.J. (2001) A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander, Biological Cybernetics, Vol. 84(5):331348. [50] Jezernik S, Colombo G, Keller T, Frueh H, Morari M (2003) Robotic Orthosis Lokomat: A Rehabilitation and Research Tool. Neuromodulation, vol.6, no.2: 108115 [51] Ju´arez-Guerrero, J., Mu˜noz-Guti´errez, S. and Mayol-Cuevas, W. W. (1998) Design of a walking machine structure using evolutionary strategies. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, San Diego, California, p. 1427-1432.
144
[52] Kagami, S., Nishiwaki, K., Kuffner, J. J., Sugihara, T., Inaba, M. and Inoue, H. (2000) Design and Implementation of Humanoid H6 and its Application to Remote Operation In: Proc. of 7th International Symposium on Exprimental Robotics. [53] Kandel, E. R., Schwartz, J. H. and Jessell, T. M. (1991) Principles of Neural Science (Third Edition) Elsevier Science Publishing Co., New York [54] Kaneko, K., Kanehiro, F., Kajita, S., Yokoyama, K., Akachi, K., Kawasaki, T., Ota, S., and Isozumi, T. (2002) Design of Prototype Humanoid Robotics Platform for HRP In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems. [55] Kato, I. and Tsuiki, H. (1972), ‘Hydraulically Powered Biped Walking Machine With A High Carrying Capacity’, Proceedings of the 4th International Symposium on External Control of Human Extremities p. 410–421. [56] Kato, T., Takanishi, A., Ishikawa, H. and Kato, I. (1981), ‘The Realization Of The Quasi Dynamic Walking By The Biped Walking Machine’, Romansy ’81 pp. 341– 351. [57] Kimura, H., Fukuoka, Y., Konaga K., Hada, Y. and Takase, K. (2001). Towards 3D Adaptive Dynamic Walking of a Quadruped Robot on Irregular Terrain by Using Neural System Model in Proc. Int. Conf. on Intelligent Robots and Systems, Maui, USA [58] Kun, A. and Miller, W. T. III Adaptive Dynamic Balance of a Biped Robot using Neural Networks (1996) Proc. IEEE Int. Conf. on RA, p. 240–245. [59] Kuroki, Y., Ishida, T., and Yamaguchi, J. (2001) A Small Biped Entertainment Robot In IEEE-RAS Int. Conf. on Humanoid Robot, pp. 181-186. [60] Lee, D. M. A., and ElMaraghy, W. H. (1992) A Neural Network Solution for Biped Gait Synthesis”, Int. Joint Conf. on Neural Networks, Vol II, p. 763-767. [61] Lee, W., Hallam, J. and Lund, H. H. (1996) A hybrid GP/GA approach for coevolving controllers and robot bodies to achieve fitness-specific tasks. In Proceedings of IEEE Third International Conference on Evolutionary Computation, Nagoya, Japan. [62] Lim, H., Setiawan, S. A. and Takanishi, A. (2002) Balance and Impedance Control for Biped Humanoid Robot Locomotion IEEE/RSJ International Conference on Robotics and Automation. [63] Lund, H. H., Hallam, J. and Lee, W. P. (1997) Evolving Robot Morphology. In Proceedings of IEEE Fourth International Conference on Evolutionary Computation NJ, IEEE Press. [64] Lund, H. H., Pagliarini, L., Paramonov, L. and Jorgensen, M. W. The VIKI Humanoids – an example of Embodied AI In Proc. 3rd Int. Symp. on Human and Artificial Intelligence Systems: The Dynamic Systems Approach for Embodiment and Sociality Fukui, Japan Dec 6-7, 2002 145
[65] Lipson, H. and Pollack, J. B. (2000) Automatic design and manufacture of artificial lifeforms. Nature, 406:974–978. [66] Mataric, M. J. (1992) Designing emergent behaviors: From local interactions to collective intelligence”, in From Animals to Animats: International Conference on Simulation of Adaptive Behavior eds. J-A. Meyer, H. Roitblat, and S. Wilson, MIT Press, 432-441 [67] Matsuoka, K. (1985). Sustained oscillations generated by mutually inhibiting neurons with adaptation. Biological Cybernetics, Vol. 52, p. 367-376. [68] Matsuoka, K. (1987). Mechanisms of frequency and pattern control in neural rhythm generators Biological Cybernetics, Vol. 56, p. 345-353. [69] McGeer, T. (1990). Passive dynamic walking. Int. J. Robotics Research, Vol. 9(2), p. 62-82. [70] McGeer, T. (1990) Passive Dynamic Walking with Knees Proceedings of IEEE International Conference on Robotics and Automation, p. 1640-1645. [71] McMahon TA (1984) Muscles, Reflexes, and Locomotion, Princeton University Press, Princeton, New Jersey [72] Miller, W. T. III (1994) Real-time neural network control of a biped walking robot, IEEE Control Systems, p. 41-48. [73] Miura, .0/2143!576 598;H. :=Aand @CBD8;EDB Shimoyama, E FHG=@ IKJMLONPQSRUTI.V;W(1984) XL
Dynamic
Walk
of
a
Biped.
[74] Miyakoshi, S., Taga, G., Kuniyoshi, Y. and Nagakubo, A. (1998). Three dimensional bipedal stepping motion using neural oscillators. In Proc IEEE/RSJ Intl Conf on Intelligent Robots and Systems, Vol 1, p. 84-89. [75] Murphy, K. and Raibert, M. (1985) Trotting and bounding in a planar two-legged model In Fifth Symposium on Theory and Practice of Robots and Manipulators, A. Morecki, G. Bianchi, K. Kedzior (eds) Cambridge: MIT Press, p. 411-420. [76] Ogihara, N. and Yamazaki, N. (2001). Generation of human bipedal locomotion by a bio-mimetic neuro-musculo-skeletal model Biological Cybernetics, Vol. 84(1), p. 1-11. [77] Paramonov, L., and Lund, H. H. (2001) A Minimalistic Approach to Humanoids. In Proc. IEEE-RAS Int. Conf. on Humanoid Robots. [78] Paul, C. and Bongard, J. C. (2001). The Road Less Travelled: Morphology in the Optimization of Biped Robot Locomotion , inProc. Int. Conf. on Intelligent Robots and Systems Maui, Hawaii, USA
146
[79] Paul, C., Iida, F., and Dravid, R. (2002) Design and Control of a Pendulum Driven Hopping Robot ”, In IEEE/RSJ International Conference on Intelligent Robots and Systems Lausanne, Switzerland. [80] Paul, C. (2003). Bilateral Decoupling in the Neural Control of Biped Locomotion , inProc. Int. Symp. Adaptive Motion of Animals and Machines Kyoto, Japan [81] Pearson, K. G. (1995) Proprioceptive regulation of locomotion. Current Opinion in Neurobiology, Vol 5, p. 786-791 [82] Pearson, K. G. and Ramirez, J. M. (1997) Sensory modulation of pattern generating circuits. In Neurons, Networks and Motor Behavior eds. P. S. G. Stein, S. Grillner, A. Selverston and D. G. Stuart. MIT Press, Cambridge, Mass, p. 225-236 [83] Pfeifer, R. and Scheier, C. (1999) Understanding Intelligence. MIT Press, Cambridge, Massachusetts. [84] Pfeifer, R., Iida, F. and Bongard, J. (2004) New Robotics: Design Principles for Intelligent Systems to appear in Artificial Life, Special Issue on New Robotics, Evolution and Embodied Cognition, MIT Press. [85] Pfeiffer, F., Loeffler, K. and Gienger, M. (2002) The Concept of Jogging JOHNNIE. In: International Conference on Robotics and Automation, p. 3129-3135. [86] Playter, R. R. (1994) Passive Dynamics in the Control of Gymnastic Maneuvers , Ph.D Thesis Department of Aeronautical and Astronautical Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts. [87] Pratt, C. A. (1995) Evidence of positive force feedback among hindlimb extensors in the intact standing cat. J Neurophysiol, Vol. 73(6), p. 2578-83. [88] Pratt, J. and Pratt, G. (1998) Exploiting natural dynamics in the control of planar bipedal walking robot. In Proceedings of the 36th Annual Allerton Conference on Communication, Control and Computing. [89] Prentice, S. D., Patla, A. E. and Stacey, D. A. (1998) Simple artificial neural network models can generate basic muscle activity patterns for human locomotion at different speeds Experimental Brain Research, Vol. 123, p. 474-480. [90] Privat, A. (2003) Repair of the mammalian central nervous system: the “spinal cord” model In: Bulletin of French National Academy of Medicine, Vol 187(2), p. 345-354. [91] Prochazka, A., Gillard, D. and Bennett, D. J. (1997) Positive force feedback control of muscles. J Neurophysiol, Vol. 77(6), p. 3226-3236. [92] Prochazka, A., Gillard, D. and Bennett, D. J. (1997) Implications of positive feedback in the control of movement. J Neurophysiol, Vol. 77(6), p. 3237-3251.
147
[93] Prochazka, A., Gritsenko, V. and Yakovenko, S. (2002) Sensory Control of Locomotion: reflexes versus higher-level control. In Sensorimotor Control, eds., S.G. Gandevia, U. Proske, & D.G. Stuart, Kluwer Academic Publishers, London, New York [94] Raibert, M. H., Brown, H. B. and Chepponis, M. (1984) Experiments in Balance with a 3D One-legged Hopping Machine In International Journal of Robotics Research, Vol 3:2, p. 75-92. [95] Raibert, M. H and Brown, H. B. (1984) Experiments in Balance with a 2D Onelegged Hopping Machine In Journal of Dynamic Systems, Measurement and Control Vol. 106, p. 75-81. [96] Raibert, M. H. (1985) Four legged-running with one legged algorithms. In Second International Symposium on Robotics Research, H. Hanafusa, H. Inoue (eds.) Cambride: MIT Press, p. 311-315. [97] Raibert, M. H. (1986) Legged Robots That Balance. MIT Press, Cambridge, Massachusetts. [98] Reil, T. (1999). Evolving bipedal locomotion in a physical simulator. Masters Thesis, University of Sussex, School of Cognitive Sciences, Brighton, UK [99] Riener, R. and Edrich, T. (1999) Identification of passive elastic joint moments in the lower extremities. Journal of Biomechanics, 32:539-544 [100] Ringrose, R. Self-Stabilizing Running (1997) In PhD Thesis Computer Science Dept, Massachusetts Institute of Technology. [101] Rossignol, S. (2000) Locomotion and its recovery after spinal injury. Current Opinion in Neurobiology. Vol. 10(6), p. 708-16. [102] Rybak, I. A., Ivashko, D. G., Prilutsky, B. I., Lewis, M. A. and Chapin, J. K. (2002). Modeling Neural Control of Locomotion: Integration of Reflex Circuits with CPG. In Proc. International Conference on Artificial Neural Networks, p. 99-104. [103] Schieppati, M. and Crenna, P. (1984) Natural cutaneous stimulation induces late and long-lasting facilitation of extensor motoneurons in the cat. Brain Research, Vol. 293(2), p. 259-267. [104] Schomburg, E.D., Petersen, N., Barajon, I. and Hultborn, H. (1998). Flexor reflex afferents reset the step cycle during fictive locomotion in the cat. Exp Brain Research Vol. 122(3), p. 339-350. [105] Seki, K. and Yamaguchi, T. (1997). Cutaneous reflex activity of the cat forelimb during fictive locomotion. Brain Research, Vol. 753(1), p. 56-62 [106] Sherrington, C. (1947) The Integrative Action of the Nervous System, 2nd ed. New Haven: Yale University Press. 148
[107] Shik, M. L., Severin, F. V. and Orlovskii, G. N. (1966). Control of walking and running by means of electric stimulation of the midbrain Biofyzika, Vol. 11, p. 659. [108] Sims, K. (1994) Evolving 3D morphology and behaviour by competition. In Artificial Life IV, p. 28-39. [109] Stein, R. B., and Capaday, C. (1988) The modulation of human reflexes during functional motor tasks. Trends in Neurosciences, Vol 11, p. 328-332. [110] Su, C. and Zheng, Y. F. (1989) An Adaptive Central Pattern Generator for Biped Robot Walking on Uneven Terrain, in Intelligent Control Systems (R. Shoureshi, Ed.), Vol. DSC-16, ASME, pp. 29–33. [111] Taga, G., Yamaguchi, Y. and Shimizu, H. (1991) Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment In Biological Cybernetics, Vol. 65, p. 147-159. [112] Taga, G. (1995) A model of the neuro-musculo-skeletal system for human locomotion. Biological Cybernetics, Vol. 73, p. 97-111. [113] Takanishi, A., Ishida, M., Yamazaki, Y., and Kato, I. (1985) The Realization of Dynamic Walking by the Biped Walking Robot WL-10RD. In Proceedings of 1985 International Conference on Advanced Robotics (ICAR’85), p. 459-466. [114] Thelen, E. and Smith, L. B. (1994) A Dynamic Systems Approach to the Development of Cognition and Action MIT Press. [115] Van Der Linde, R. Q. (1999) Design, analysis and control of a low power joint for walking robots, by phasic activation of McKibben muscles In: IEEE Trans. Robotics and Automation, Vol. 15(4), p. 599-604. [116] Vukobratovic, M. and Juricie, D. (1969) Contribution to the Synthesis of Biped Gait In IEEE Tran. On Bio-Medical Engineering, Vol. 16(1), p. 1-6. [117] Wadden, T. and Ekeberg, O. (1998) A neuro-mechanical model of legged locomotion: single leg control In Biological Cybernetics, Vol. 79, p. 161-173. [118] Williamson, M. M. (1998) Rhythmic Robot Arm Control Using Oscillators, In Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems Victoria, p. 77-83. [119] Winter, D. (1990) Biomechanics and motor control of human movement, second edition. J Wiley and Sons, New York. [120] Wisse, M. and van Frankenhuyzen, J. (2003). Design and construction of MIKE; a 2D autonomous biped based on passive dynamic walking , inProc. Int. Symp. Adaptive Motion of Animals and Machines Kyoto, Japan.
149
[121] Yamaguchi, J., Soga, E., Inoue, S. and Takanishi, A. (1999) Development of a Bipedal Humanoid Robot - Control Method of Whole Body Cooperative Dynamic Biped Walking, In Proc. IEEE Int. Conference on Robotics and Automation, p. 368-374. [122] Zajac FE (1989) Muscle and Tendon: properties, models, scaling and applications to biomechanics and motor control CRC Crit. Rev. Biomed. Eng., 17:359-419 [123] Zheng, Y. and Shen, J. (1990) Gait synthesis for the SD-2 biped robot to climb sloped surface, IEEE Trans. Robot. Automation 6(1):86-96.
150
