This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS

1

Nullcline-Based Design of a Silicon Neuron Arindam Basu, Student Member, IEEE, and Paul E. Hasler, Senior Member, IEEE

Abstract—In this paper, we describe the design of a silicon neuron that exhibits type-I neural excitability, i.e., the frequency of spiking of the neuron approaches arbitrarily close to zero as the input current is reduced. Our design creates a conductance-based silicon model that can exhibit a saddle-node bifurcation. We present simulations and measured data from circuits fabricated in 0.35- m CMOS that demonstrate both saddle-node bifurcation on invariant circle and saddle-homoclinic bifurcation. In our design, concepts from nonlinear dynamics are used not only for the analysis but also for the synthesis of the circuit. This leads to a nullcline-based methodology that enables a strategic approach for biasing the circuit in the desired regime in parameter space. Combined with the ability to set local biases (e.g., floating gates), this methodology should largely minimize mismatch in arrays of silicon neurons of this kind. The presented circuit is the most power efficient design reported so far, and we hope to fabricate larger arrays of this neuron to explore network behavior. .Index Terms—Ion-channel dynamics, nullcline, saddle-node bifurcation, silicon neuron, type-I membrane

I. MODELING BIOLOGICAL NEURONS IN SILICON EUROMORPHIC systems emulate biological structures in the hope of retaining its remarkable computational efficiency. These artificial systems that can replicate neural behavior in real time would be useful to neuroscientists for exploring neural computation, as well as engineers who want to build power efficient processors for various applications. Another advantage of having real-time computation is that these artificial devices may be used for directly interfacing with biology in a dynamic clamp system or can be even used to replace nonfunctional neurons as a prosthetic. Recent findings have underlined the importance of spiking neuron models as compared to the long-studied static neural networks. For example, there have been a lot of evidences suggesting the existence of spike-timing-dependent plasticity, which is a Hebbian-type learning rule in the cortex that depends on exact spike timings. This necessitates the study of spiking neural models for learning tasks like memory formation and pattern recovery [1]–[3]. In the classification of neural spiking proposed by Hodgkin [4], he classified neurons in three broad categories. The first two were capable of repetitive spiking. Type-I neurons possessed (frequency–current) curves that approached zero, i.e., the

N

Manuscript received November 25, 2009; revised February 05, 2010; accepted March 25, 2010. This paper was recommended by Associate Editor A. van Schaik. A. Basu is with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798 (e-mail: arindam.basu@gmail. com). P. Hasler is with the School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332-250 USA (e-mail: phasler@ece. gatech.edu). Digital Object Identifier 10.1109/TCSI.2010.2048772

Fig. 1. Silicon neuron design: (a) The methodology for creating silicon neurons that we use involves designing for certain desired bifurcations. (b) Our concept of silicon neurons has transistors modeling biological channels (sodium, potassium, etc.) with amplifier circuits sensing the membrane voltage and changing the conductance of the channel transistors appropriately.

neurons were capable of firing at large, as well as arbitrarily low, frequencies, while type-II neurons transitioned from silence to firing at an arbitrary nonzero frequency. Subsequent analysis, put forward in [5] and summarized in [6], has shown that these two mechanisms ubiquitously describe nearly all spiking neuron models. Furthermore, the type-I characteristic is uniquely associated with a saddle-node bifurcation at the transition from silence to spiking, and the oscillation (spiking) responds to depolarizing perturbations with a phase advance. The type-II characteristic is uniquely associated with a Hopf bifurcation, and the oscillation (spiking) responds to a depolarizing perturbation by either advancing or delaying the oscillation, depending on where in the limit cycle the input arrived. These two types of neurons have different synchronization properties to which can be studied by analyzing their phase response curves. Since synchronization is considered as one of the computational primitives of neurons that is useful in object binding [3] or attention [7], it can be argued that these two types of neurons perform different computational tasks. All the current silicon designs typically use a neuron exhibiting class-I behavior. Most of these designs have used the integrate and fire (I&F) model and its variants because of its simplicity. However, these phenomenological models fail to capture many properties of actual conductance-based neurons like phase response curves. On the other hand, some designs have faithfully replicated full Hodgkin–Huxley (H-H) dynamics [8]–[10], resulting in large footprints for each neuron. Our silicon neuron with class-I membrane dynamics lies in a space between these two. Fig. 1(a) shows the basic philosophy: The differential equations modeling ion-channel dynamics in a neuron exhibit certain bifurcations which are responsible for its computational properties [6]. We make circuits that exhibit the same bifurcations but do not necessarily model the original differential equations. This allows us to create more compact

1549-8328/$26.00 © 2010 IEEE Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 2

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS

Fig. 2. Transformation from Hopf neuron: (a) Removing the transistor responsible for inactivation of the sodium current M3 results in a persistent sodium is increased to allow for approximating sodium activation current which can produce multiple equilibria. A leak channel is added to create the cubic nullcline. I as instantaneous. The modified elements are encircled. (b) In this figure, x and n signify normalized membrane potential and potassium activation, respectively. Nullclines of the new neuron structure shows three equilibria, two of which are stable and one is unstable.

and power efficient designs. Fig. 1(b) shows a schematic of the neuron where a transistor is used to model the biological channel while a separate circuit senses the membrane voltage and applies the appropriate voltage on the gate of the channel transistor to modulate its conductance. The feedback circuit can be designed based on the desired bifurcation sequence (which implies a certain set of associated phenomena like positive feedback and phase response curve [6]) or on the voltage clamp responses of individual channel modules. An earlier paper [11] demonstrated class-II excitability using a similar approach. Here, we derive the circuit for class-I membrane dynamics starting from the earlier design in [11] and [12] and also develop methods for algorithmically biasing the circuit in the correct regime based on nullclines (curves along which the vector field is zero horizontally or vertically) of the reduced 2-D model. The circuit we present for extracting the nullclines, coupled with a method for setting local biases [e.g., floating gates (FGs)] in a neuron array, shall also allow us to reduce mismatch-induced variations in the neuron array. Takemoto et al. [13] report a bifurcation-based analysis of a class-I neuron, but that circuit is an order of magnitude more in transistor count compared to the proposed one. Moreover, the authors only report a discrete implementation without considering the effects of mismatch. Moreover, in that design, nullclines are used only for analysis and not for synthesis of the circuit. In Section II, we describe the derivation of the neuron circuit and formulate the differential equations governing its dynamics. We also present its phase portrait and bifurcation diagrams for different parameter values. Section III deals with extending the circuit to measure the nullclines and, hence, methodically bias the circuit. In Section IV, we present the results from an IC and describe the current–frequency curve and measured nullclines. We also describe spike frequency adaptation in this neuron by adding another slow potassium channel. Finally, we compare our work with others and present the conclusions in the last section. II. CIRCUIT OPERATION AND MODEL In this section, we start from the Hopf neuron model presented earlier [11] and follow a route of reducing this model

to one that can exhibit saddle-node bifurcations (we call it a “saddle-node neuron”). We deduce theoretically that changes have to be made to the -channel structure to get the desired behavior. The existence of limit cycles can also be understood by studying the phase portraits of the model. This approach of studying bifurcations is useful because it is believed that computational properties of neurons are based on the bifurcations exhibited by these dynamical systems in response to some changing stimulus [5], [6]. For example, any neuron exhibiting a Hopf bifurcation can easily signal when a stimulus crosses a threshold by initiating a spike train, while those exhibiting saddle-node bifurcations can encode the strength of a stimulus in their firing rate. Hence, by showing that this silicon neuron has bifurcations similar to a certain class of biological neurons, we can claim that the silicon neuron can also perform similar computations. A. Hopf Neuron and Its Modification The circuit on the left in Fig. 2(a) shows the Hopf neuron that we have presented earlier. We consider these circuits to be composed of individual channels. Each channel has a channel transistor that supplies the current and a gating amplifier that controls the gate of this transistor in some nonlinear way to produce desired dynamics. This circuit has only one equilibrium [11] because the gating amplifier for the transient sodium channel is bandpass leading to the sodium-channel transistor acting as approximately a constant current source in steady state. Hence, the - relation for this neuron is dictated by the monotonic exponential - relation of the potassium channel transistor, leading to only one equilibrium. 1) Absence of Inactivation: In order to create a saddle-node neuron, there must be multiple equilibria, one of which should be of the saddle type. Thus, the neuron must have a nonmonotonic - curve with multiple zero crossings. To create a nonmonotonic curve, we must have a positive feedback channel that creates a negative conductance region (since there has to be a part of the - curve where current increases even though the voltage across the channel decreases). This implies that the positive feedback sodium channel in this implementation needs to be a low-pass one instead of a bandpass one so that it can affect

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. BASU AND HASLER: NULLCLINE-BASED DESIGN OF A SILICON NEURON

3

the dc/persistent current of the channel. This model is typically called “persistent sodium” model as compared to the HH model which is a “transient sodium” model. Thus, the feedback tranneeds to be removed, sistor providing inactivation current as shown in the right half of Fig. 2(a). The fact that the low-pass amplifier does indeed give saddle-node bifurcation can also be seen from the equations in [11] by a change of variables and let. Essentially, a simple common source amplifier or ting any other dc-coupled amplifier may also be used in this place. 2) Instantaneous Activation: The typical way to simplify the model further [6] is to assume that the sodium activation is much faster than the potassium; therefore, it can be approximated to be instantaneous. Since a decrease in the gate voltage of is responsible for the activation of sodium current, the earlier , where is assumption can be obtained by setting any other bias current in the circuit, so that the rate of change of is much faster than that of . Thus, we have the gate of a 2-D system with the membrane potential and the voltage at the gate of the potassium channel transistor being the only two variables. The other major difference from the Hopf neuron is the need for a leak channel, as shown in Fig. 2(a). This is needed to create and control the increasing part of the nullcline for low membrane potentials, as shown in Fig. 2(b). The net current from is the leak and sodium channels becomes negative after , leading the gate voltage of the potassium slightly larger than amplifier (the nullcline) to tend to infinity, creating the initial rising part of the nullcline. This can be controlled by , which is a feature that will be used later. Assuming a hyperbolic-tangent (tanh) function to approximate the amplifier’s characteristic, applying Kirchoff’s current law to the neuron in a way similar to [11] yields

(1) where

, , , , , and the derivatives are taken with respect , to . All the currents in the equation are normalized by . Moreover, paramewhile capacitors are normalized by ters “ ,” “ ” and “ ” represent the input current stimulus, the amplifier’s trip point, and gain, respectively. It should be noted comes from the ohmic nature of and that the term . The coubecause of the assumption that pling of a transistor’s gate to the surface potential has also been assumed to be one. This is not a critical assumption since it will be shown later that the nullclines obtained from the actual circuit are very similar to the ones obtained numerically with this assumption. The phase portrait for this system is shown in Fig. 2(b). The variables and signify normalized membrane voltage and potassium channel activation, respectively. B. Motivation for Changing the Potassium Channel Amplifier It can be seen from Fig. 2(b) that the -nullcline now has three branches (a typical “N” shape). Thus, there is a possibility

Fig. 3. Saddle neuron: A compact silicon neuron that exhibits saddle-node bifurcations of the equilibrium. The limit cycle might be born by SNIC or an SHM loop bifurcation. (a) and (b) show the non-FG and FG versions, respectively. Spike frequency adaptation can be obtained by adding another similar potassium channel with a slower activation time constant set by 3.

M

of multiple intersections with the other nullcline and, hence, the existence of multiple equilibria. However, the equation for . the -nullcline in (1) is linear and can be written as Hence, the -nullcline will intersect the -nullcline in all three branches. However, it is well known [6] that most of the middle branch of the -nullcline in this type of a system corresponds to unstable equilibria, while the other two branches indicate stable equilibria. Therefore, in this case, we will have a stable equilibria at a high value of membrane potential for a large range of parameter values, something not commonly observed in biology. The existence of this equilibria might also be understood intuitively. At low membrane potential values, the sodium current will be balanced by a leak current and the potassium current. As the membrane potential increases, the sodium current increases a lot but then starts to decrease when the sodium-channel . At the same time, the potasamplifier saturates at sium channel current is increasing, and hence, there should exist , where they are equal and perturbations a high value for to that state also die down. We want to have a saddle node on invariant circle (SNIC) or a saddle-homoclinic (SHM) loop bifurcation with increasing input current since these are the two commonly observed saddlenode-related bifurcations in neurons. In most neural models, the limit cycle born from these bifurcations is the only stable attractor [5], [6]. This implies that there should not be any other stable equilibria when the saddle-node bifurcation involving the equilibrium at low membrane potential occurs. Thus, the -nullcline has to intersect the -nullcline once in the lowest membrane potential branch and twice in the middle branch, implying that it must have a sharply falling part. Hence, there needs to be with the desired an inverting amplifier that drives the gate of characteristic. This leads to a circuit for the saddle neuron, as shown in Fig. 3. Both the FG and non-FG versions are shown. The FG version has not yet been fabricated; all measured results presented in this paper are from the non-FG version, while only simulations are presented for the FG version. The FG version has the advantage of requiring two less biases and can be more compact without trading off transistor threshold voltage matching. It will be shown in Section IV that using only three FG transistors, as shown in Fig. 3(b), is sufficient to eliminate

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 4

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS

the sources of mismatch in the circuit. In this circuit, and form the Na-channel inverting low-pass amplifier, while and form the -channel inverting low-pass amplifier. In the FG version, capacitors (which can be explicitly drawn capacitors or overlap capacitors associated with the transistors) set the gain of the gating amplifiers. To ensure that the -nullcline can intersect the -nullcline twice in the middle branch, the gain of the potassium amplifier has to be larger than the sodium is chosen large enough so that the activation kiamplifier. netics of both the amplifiers are faster than other timescales in . the circuit. The slow -channel timescale is controlled by We could have used a PMOS or a resistor in its place, which would result in slightly different shapes of the spikes. The equations to model this circuit are as follows:

(2) with similar interpretation of variables as in (1). Parameters and can be modified by changing and in the non-FG version or by changing the charge on the gates of and , respectively. and are set by the difference in the power supply voltages of the amplifiers. Spike frequency adaptation can be obtained in this neuron by adding another similar potassium channel but with slower activation kinetics compared to the potassium channel responsible for regular spiking. It should be noted that (2) is similar to the traditional Morris–Lecar model [14], which is obtained by following a similar reduction principle from the H-H model. The differences between (2) and the Morris-Lecar equations stem from using transistors with exponential characteristics in place of channels with linear characteristics. C. Phase Portrait and Bifurcation Diagrams In this section, the numerical simulations of (2) are described. Bifurcation diagrams for varying input current are computed using the continuation software AUTO. Two different cases are considered based on the speed of the potassium activation. The parameters for the numerical simulation results shown in Fig. 4 are presented in Table I. Fig. 4(a) shows the phase portrait of the system with a relatively slow potassium activation time constant (smaller ) and a fixed value of input current that is close to the bifurcation value. The nullclines of the system shown in the figure do not change with the activation time constant. There are three intersections of the two nullclines, as desired, leading to two unstable and one stable equilibra. The middle equilibrium is a saddle node and is about to collide with the stable equilibrium to create a bifurcation. It can be seen that the trajectory starting from the right of the saddle makes a big loop before returning to

Fig. 4. Phase portrait and bifurcation diagram. (a) Phase plane for the system with slow activation of -channel. The -nullcline has three distinct regions. In region A, changes from lower than to larger than it resulting in leak current overpowering sodium current. In region B, is close to , . It can while in region C, the sodium amplifier has saturated at be seen that the trajectory starting from the right of the saddle makes a big loop before returning to the stable equilibrium. (b) Phase plane for fast potassium activation, leading to SHM loop bifurcation. Even before the saddle-node bifurcation, a limit cycle has been created by a homoclinic bifurcation from the saddle equilibrium. (c) and (d) Bifurcation diagram for the case in (a) and (b).

x

K

x E

o x Na amp vss

TABLE I PARAMETERS FOR NUMERICAL SIMULATION

the stable equilibrium. At the saddle-node bifurcation, this trajectory becomes homoclinic to the saddle, and after the bifurcation forms a stable limit cycle. As the homoclinic loop is an invariant manifold homeomorphic to a circle, this is also called SNIC. From (2), it can be seen that the -nullcline is given by , as shown in Fig. 4(a) or (b). To study the structure of the -nullcline, one must understand that the -axis correquired to make responds to the voltage on the gate of , i.e., to balance the rest of the current incident on the membrane capacitor. The nullcline can be divided into three , distinct zones, marked as A, B, and C in Fig. 4(a). For there exists a value of to balance the leak and sodium currents. However, for sufficiently larger than , such that the leak current is larger than the sodium current, no value of exists since the polarity of the leak current is then similar to the potassium current. Thus, the -nullcline tends to infinity near the end of zone A. In zone B, is close to , and the sodium control amplifier is in the high-gain region. The slope of the , where is the gain nullcline in this region is of the sodium amplifier and is the coupling strength of the

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. BASU AND HASLER: NULLCLINE-BASED DESIGN OF A SILICON NEURON

5

gate to surface potential for a transistor. Lastly, in zone C, the , implying that sodium amplifier has saturated at changes to compensate for the change in potassium current due to the change in membrane potential. Therefore, the slope . The nullcline changes sharply at the ends in this region is or is in the ohmic region. when Fig. 4(c) shows the bifurcation diagram in this case, which shows the birth of a limit cycle coinciding with a saddle-node bifurcation. The limit cycle ends in a supercritical Hopf bifurcation for larger values of input current. Fig. 4(b) shows the phase portrait for the case where the activation of potassium is relatively fast. As can be seen from Fig. 4(b) and (d), even before the saddle-node bifurcation, a limit cycle has been created by a homoclinic bifurcation from the saddle equilibrium. The main observable difference from SNIC is the bistability of the system before the saddle-node bifurcation in a certain parameter range. Intuitively, this is because the potassium current deactivates quickly and does not allow the membrane potential to undershoot the stable equilibrium voltage. The limit cycle, in this case, ends in a subcritical Hopf bifurcation at a larger current value. III. NULLCLINE EXTRACTION AND ALGORITHMIC BIASING The last section described the importance of nullclines in determining the dynamics of this circuit. Hence, it is of utmost importance to be able to extract the nullclines of the fabricated system and then vary the parameters to get the right shape and intersections. In this section, we propose a circuit to do so and also describe an algorithm for biasing the circuit based on the measured nullclines. A. Nullcline Measurement Circuit Fig. 5(a) and (b) shows the methods to measure the - and -nullclines of this system. Note from (1) that and denote normalized values of the membrane voltage and potassium channel gating voltage, respectively. As mentioned earlier, the -nullcline of the system is given by since that is the value of for which the current charging is zero. This is exactly equal to the characteristic of the potassium gating amplifier. Hence, we only need to monitor the output voltage of the potassium amplifier to get this nullcline. such that on this The -nullcline is a curve curve. Finding the value of on this curve corresponding to a (say) is equivalent to forcing the decertain value of sired using an amplifier connected in feedback controlling the gate of . The desired “ ” value is created at the output of the amplifier. This structure is similar to a logarithmic amplifier that we have used earlier for measuring low currents [15]. However, one can see from Fig. 5(b) that there is an extra NMOS, Mn also connected in feedback across the amplifier together with . Here, it is necessary to prevent the operating point of the circuit to be attracted to an undesired fixed point. A natural question to explore is the area penalty for including this circuit with a neural array. Since it is desirable to maximize the number of neurons in a chip, requiring an amplifier in every neuron would be prohibitive. Fortunately, it is possible to share one nullcline-measuring circuit for an entire array of neurons. Also, this eliminates the possibility of adding any mismatch due

Fig. 5. Circuits to extract the nullcline: (a) n-nullcline for the system is a curve n (x) such that dn=dt = 0 on this curve. This is the same as the characteristic of the K -amplifier. (b) x-nullcline is a curve n (x) such that dx=dt = 0 on this curve. Finding the value of n on this curve corresponding to a certain value of x = V (say) is equivalent to forcing the desired x using an amplifier connected in feedback controlling the gate of M . (c) Modification for integrating these neurons into an array with the selection, biasing, and nullcline measurement circuitry at the periphery.

to having different nullcline-measuring circuits. Fig. 5(c) shows the modifications needed to enable this for the th neuron in an array. A control signal puts the neuron array in nullcline-measuring or normal operational mode. Apart from the common biasing of the neurons, three other lines are shared , , and . Switches controlled among the neurons: by a decoder are introduced in every neuron so that the mem, and the output of the potasbrane potential, the gate of sium gating amplifier are connected to those three global lines in the nullcline-measuring mode. The amplifier and other selection circuits can now be confined to the nullcline measurement block at the periphery of the array. B. Algorithmic Biasing Using the circuits described earlier, we can extract the nullcline of the circuit. Now, we describe a methodical approach for setting the biases for the non-FG circuit based on the measured nullclines with a goal of obtaining a saddle-node bifurcation with increasing current. The FG-based circuit can be biased in a similar way. , , , 1) In the beginning, , and are set to to shut off the act as an ON switch. gating amplifiers and make is set to zero to turn off the leak channel, too. The difference between and is a system level specification, typical values being 200–300 mV. The absolute values of these bias voltages are kept close to the middle of the chip’s power supply to allow for sufficient headroom in setting the gating amplifiers’ power supplies.

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 6

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS

TABLE II PARAMETERS FOR THE FABRICATED CIRCUIT

Fig. 6. Die photo and layout: (a) Die photo of the fabricated FPAA in 0.35-m CMOS showing a CAB with low-pass sodium and potassium channels. (b) Layout of the optimized FG-based neuron cell for a custom chip in the same process.

2) Set to the value of , where it is desired for zone A of the -nullcline to end. is initially set at . is set 3) The value of so that is large enough to set a time constant much faster than that of the slow potassium (typical values are much higher ini0.5–1 nA). A good strategy is to set until the spike shape changes. tially and then reduce 4) is then reduced gradually from . This shifts zone B of the -nullcline to progressively lower voltages. Continue until zone B is in the desired range of memis set to brane potentials. For the FG neuron, around 1 V below , while zone B is set by modi. fying the charge on the gate of until the -nullcline has the characteristic of 5) Increase zone A and increases sharply to a high voltage. Increase by approximately 10–20 mV to ensure a high enough sets the threshold current (exthreshold current. Since ponential relation), its final value can be set after measuring the graph of its dependence. is reduced to approximately 50–100 mV 6) below the lowest value of the measured -nullcline. is now set using a strategy similar to the setting of for a particular . 7) is slowly reduced from . This shifts the -nullcline to lower membrane potentials. Continue until the -nullcline crosses the -nullcline in zone B. For the FG circuit, the -nullcline is shifted by modifying the charge on the gate of . To verify that the circuit does have a limit cycle, it can be configured as neuron and input current applied to check for oscillations. IV. MEASUREMENTS The proposed neuron (non-FG) and related concepts have been verified by fabricating an IC with the neuron circuit in 0.35- m CMOS process, which is shown in Fig. 6(a). The fabricated IC has an architecture similar to a field-programmable analog array (FPAA), as described in [16]. The elements in the computational analog blocks (CABs) included operational transconductance amplifiers, synapses, and different types of channels. The switches needed for configuring the neuron into operational mode or nullcline measurement mode, as described in Section III, were not explicitly needed since the reconfigurable switch matrix could perform that task. Since the neuron was placed in a CAB, it was not optimized for size and also

included individual biasing circuits. The resulting area is 2740 m , including the extra potassium channel for spike frequency adaptation. The poly–poly capacitor took up a major share of the area, which is an aspect that can be largely improved by using a MOS capacitor instead. The transistor sizes and bias voltages are tabulated in Table II. Note that the channel length of transistors in the potassium gating amplifier was made much larger than the sodium amplifier to make it have a larger gain, as noted in Section II. An optimized layout of the neuron with MOS capacitors replacing poly-poly ones and including an extra potassium channel for spike frequency adaptation has now been done for a custom chip, as shown in Fig. 6(b). This FG-based neuron . occupies a much smaller area of A. Nullclines, SNIC Bifurcations, and Power Dissipation Fig. 7(a) shows the measured -nullclines from the fabricated reduces the trip point of chip. As expected, reducing the potassium amplifier (or ), resulting in the high-gain part of the activation curve, moving to lower membrane potentials. Fig. 7(d) shows the measured -nullcline for different values of input current. As mentioned earlier, the sharply rising part at the end of zone A in Fig. 4(a) is because of the leak current overpowering the sodium current. However, an increase in the input current can balance the leak, as can be seen in the figure. This confirms the existence of the saddle node when the channels are combined together. Fig. 7(b) and (e) shows the measured spiking waveforms from the neuron. Fig. 7(b) corresponds to a lower value of the input current that is slightly larger than the bifurcation value. The variability in the interspike interval is due to ambient noise. The shape of the spikes is typical of SNIC bifurcations and type-I membrane dynamics. In the period right after a spike, there is a hyperpolarization of the membrane followed by a long time when the membrane appears to be at equilibrium. In this time, the solution is passing through the region of the phase space where the saddle node existed earlier. The resulting small vector fields lead to the sluggishness of the trajectory. Fig. 7(c) shows the relation between the threshold current for repetitive spiking and or the parameter setting the leak conductance. The relation is approximately exponential, suggesting that the bifurcation occurs almost when the input current balances the leak current. The solid curve in Fig. 7(f)

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. BASU AND HASLER: NULLCLINE-BASED DESIGN OF A SILICON NEURON

7

Fig. 7. Measured results: (a) Variation of the n-nullcline with decreasing V shows a decrease in o . (d) Increasing the input current stimulus, resulting in variation of the x-nullcline. The existence of a saddle-node bifurcation can be inferred from the disappearance of the sharply rising part of the nullcline at the end of zone A in Fig. 4(a). (b) and (e) Measured spiking waveforms from the neuron for two different input current values. (c) Threshold current for repetitive spiking (bifurcation value) can be directly controlled using V . The resulting variation is exponential. (f) Measured (solid) f I curve and (dashed) power dissipation of the neuron as a function of spike frequency. Its power consumption of 1.74 nW at a spike frequency of 100 Hz is the lowest reported to date.

0

shows the measured current-frequency behavior ( curve) of . It exhibits a typical subthe neuron for a certain value of linear behavior (theoretically square root in the neighborhood of the bifurcation). It should be noted that spike frequency adaptation was not included for this measurement. A major motivation in developing this design was to achieve the lowest possible power dissipation. For the neuron with spike frequency adaptation, there are three gating amplifiers, each of which was biased at a current of around 0.5 nA. The other component of the power dissipation is the switching power which has been measured for several spike frequencies. The resulting power dissipation plot is the dashed curve in Fig. 7(f). As expected, the switching power increases almost linearly with frequency. It should also be noted that the switching power is not the dominant factor in the net power dissipation. It may be possible to reduce the static power even further since the spike shape was not distorted until this point. With a power consumption of 1.74 nW at a frequency of 100 Hz, this neuron has the lowest power consumption among all reported designs to the best of the authors’ knowledge. The new design of the neuron that shares the gating amplifier for both the potassium channels is expected to reduce power consumption further. B. Spike Frequency Adaptation A common feature of neurons that has been widely modeled is spike frequency adaptation. It was mentioned in Section II

that this phenomenon could be observed in this neuron model by adding a slower potassium channel along with the regular ones. This channel models a calcium-gated potassium channel. The reconfigurability of the FPAA platform allowed us to verify this concept easily. The biases for this potassium amplifier were kept the same as the regular potassium amplifier. The activation time constant for this channel was increased by reducing by 250 mV compared to the regular potassium channel. The input current to the neuron was stepped from a value lesser than the bifurcation value to one where the neuron spikes. The resulting waveform is shown in Fig. 8. It clearly displays increasing interspike interval as it continues spiking. To decrease power consumption and chip area, the new design of the neuron shares the potassium amplifier for both the slow and fast potassium channels. C. Mismatch and Variability A major concern in the design of arrays of spiking silicon neurons is mismatch among the neurons. The two parameters, whose matching we are concerned with in this design, are the curve and the bifurcation current value corresponding to the birth of the limit cycle. Since biases are shared across neurons in the array shown in Fig. 5(c), the matching of the neuron parameters would require very large sized transistors, increasing the footprint of one neuron. Instead, we plan to use FG transistors, as shown in Fig. 3(b), that decouple the constraint of transistor size dictating the mismatch. Of course, a programming circuit

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 8

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS

Fig. 8. Spike frequency adaptation: Adding an extra potassium channel that has slower kinetics than the one responsible for regular spiking results in spike frequency adaptation. The measured waveform is the neuron’s response to a current step.

is now needed. However, since we can share one such circuit for the entire chip, it is not prohibitive. A naive solution would be to replace all the transistors in the circuit with FGs, but this would result in a severe area penalty because of the added selection circuitry per FG transistor in each cell. Instead, an analysis of this neuron circuit reveals that using only three FG transistors, as shown in Fig. 3(b), allows us to recurve and the curmove mismatch-induced variations in the rent threshold for repetitive spiking (bifurcation current). First, to control the nullclines, the ability to tune the half-activation and voltages of both gating amplifiers is needed. Hence, need to be FGs. This also makes the gain of the amplifiers relatively insensitive to mismatch as they are now set by capacsets the threshold current itor ratios. As shown in Fig. 7(c), and hence needs to be an FG transistor. Variations in threshold only shift the -nullcline up or down in the region C, of which is a phenomenon whose effect is nullified by ensuring that the -nullcline does not intersect the -nullcline in that part (by keeping low enough). and and the capacitor The remaining two transistors affect the slope of the curve directly. To understand their effect, we need to analyze the components of a spike interval. The spike itself takes much less time compared to the to time for the membrane to charge from its reset voltage . For a simplified analysis, we the threshold for a spike can consider the time period to be determined by linear charging of a membrane capacitor by the difference between input and leak currents. This is of course a simplification, as we ignore the ohmic nature of the transistors. However, it does allow us to is set by the location of zone B, capture the trend easily. i.e., the half-activation potential of the sodium channel. Hence, the time period of oscillation for a certain input current is given by (3) is the voltage that the membrane is reset to after a where increases for an increase in or a decrease in spike. . Fig. 9 shows the simulated effect of a 30-mV increase in compared to the nominal case and the resulting variation curve. From (3), it is obvious that increasing in the

M

Fig. 9. Mismatch correction: Threshold voltage variation of 30 mV in 3 causes a large change in the f i curve. The slope can be corrected by changing the either the voltage threshold of the neuron or the synaptic efficacy. The current threshold for repetitive spiking is modified by changing the leak current.

0

Fig. 10. Monte Carlo analysis: A Monte Carlo analysis is performed on the neuron circuit assuming a standard deviation of 1 mV for the transistor threshold voltages (assuming FG transistors are used to correct for the large threshold variations). The resulting f i curve is plotted. EKV models were used for the simulation.

0

(by modifying the charge on the gate of in Fig. 3(b)) approcurve. This would howpriately can correct the slope of the ever result in a change in the bifurcation current or the origin of curve. The leak current is then changed to bring the bithe furcation current close to the desired value. The corresponding corrected curves are also plotted in the figure. Another possible curve method for eliminating mismatch in the slope of the in a network of these neurons is modifying the synaptic efficacies of FG synapses connected to this neuron. Having shown that the three FG transistors in Fig. 3(b) can eliminate mismatch if its charge is modified appropriately, we need to estimate the sensitivity of the neuron’s response to the programmed charge. Hence, the effect of finite resolution in programming the FG charge is discussed next. We have earlier accuracy shown the ability to program FG currents to a of 0.1% over three to four decades of current [20] or to 1% over six to seven decades [16]. An accuracy of 1% in currents corresponds to an accuracy of 0.36 mV, assuming subthreshold operation and that threshold mismatch is dominant among all the sources of variation. To consider an extreme worst case scenario, we considered a 1-mV standard deviation in threshold voltages and performed a Monte Carlo analysis on the neuron’s curve. For each value of input current, 100 runs were performed to obtain the mean and variance of the spike frequency.

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. BASU AND HASLER: NULLCLINE-BASED DESIGN OF A SILICON NEURON

9

TABLE III PERFORMANCE COMPARISON OF SILICON NEURONS

SFA: Spike frequency adaptation; PF: Positive feedback; RP: Refractory period FPAA-based design Custom chip (I; V ) mode: (Current, voltage) mode Assuming two conductances are needed for a basic neuron Also includes three synapses

Enz-Krummenacher–Vittoz (EKV) models were used for the simulation. The results are shown in Fig. 10. The resulting varicurve is acceptable, keeping in mind that this ation in the is a conservative estimate.

The figure of merit (FOM) quoted in the table is a metric describing the complexity of the model, its power dissipation, and normalized area. It is given by the following equation: (4)

V. CONCLUSION Spurred by an increasing interest in real-time simulation of spiking neural networks, many researchers have developed integrated circuits modeling neurons over the last few years. These neurons have varied in detail from full H-H [8], [9] to simple I&F [1] models. The full H-H implementations have mostly suffered from excessive power dissipation and area penalties, while I&F models often do not possess many important dynamics (e.g., realistic phase response curves leading to synchronization). This has led researchers to develop hybrid models which strike a compromise and mimic certain properties that are most desirable to the user in their application. Spike frequency adaptation, refractory period, positive feedback, and conductancebased models are common examples of such properties [3], [17], [18]. Common implementation methodologies for these neurons are based on switched capacitors or current-mode design. While switched-capacitor-based methods offer good matching, they suffer from a tradeoff between area and programmability. Current-mode designs, on the other hand, are limited by mismatch and are difficult to bias. A comparison of several representative designs is presented in Table III. The power consumption for a biological neuron is an approximate number obtained from [21], assuming that the 20 W of power dissipated in the brain is divided equally among its thousand billion neurons.

is the minimum channel length in the VLSI process where is the power dissipation used, is the area of the neuron, is a measure of the model for a spike rate of 100 Hz, and detail. For different applications, some of these metrics (e.g., power) may be more important than others; then, that factor may be weighed more to arrive at a desired FOM. “ ” can be chosen in different ways, depending on the application. In this case, we wanted to use this to represent the variety of dynamical behaviors exhibited by the model. Hence, we chose it to be a fraction which is determined by the number of features out of the 20 listed in [22] that is exhibited by that particular model. It should be noted that the detailed physics of an ion channel or biologically realistic spike shape might be extremely important in some applications (e.g., dynamic clamps); this feature has not been taken as a metric here. In such a scenario, a more appropriate choice of may be the number of digital FLOPs needed to compute the model currents since more realistic models (like H-H) require significantly more computation. We have presented a bifurcation-based silicon neuron that consumes the lowest power among all designs reported to date. We have also proposed a circuit to extract the nullclines for the system, which leads to an algorithm for correctly biasing the circuit. Combined with the ability of FG transistors to set bias

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 10

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS

voltages locally, this method should reduce variation in silicon neurons while not incurring a significant area penalty.

ACKNOWLEDGMENT The authors would like to thank Dr. R. Butera for the many helpful comments and suggestions.

REFERENCES [1] G. Indiveri, E. Chicca, and R. Douglas, “A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity,” IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 211–221, Jan. 2006. [2] R. Vogelstein, U. Mallik, J. Vogelstein, and G. Cauwenberghs, “Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses,” IEEE Trans. Neural Netw., vol. 18, no. 1, pp. 253–265, Jan. 2007. [3] J. Arthur and K. Boahen, “Synchrony in silicon: The gamma rhythm,” IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1815–1825, Nov. 2007. [4] A. L. Hodgkin, “The local electric changes associated with repetitive action in a non-medullated axon,” J. Physiol., vol. 107, no. 2, pp. 165–181, Mar. 1948. [5] J. Rinzel and G. B. Ermentrout, Analysis of Neural Excitability and Oscillations, Methods in Neural Engineering, C. Koch and I. Segev, Eds. Cambridge, MA: MIT Press, 1989. [6] E. M. Izhikevich, Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting. Cambridge, MA: MIT Press, 2007. [7] A. Roy, P. Steinmetz, S. Hsiao, K. Johnson, and E. Niebur, “Synchrony: A neural correlate of somatosensory attention,” J. Neurophysiol., vol. 98, pp. 1645–1661, Jun. 2007. [8] S. Saighi, J. Tomas, Y. Bornat, and S. Renaud, “A conductance-based silicon neuron with dynamically tunable model parameters,” in Proc. Int. IEEE EMBS Conf. Neural Eng., 2005, pp. 285–288. [9] M. F. Simoni, G. S. Cymbalyuk, M. E. Sorensen, R. L. Calabrese, and S. P. DeWeerth, “A multiconductance silicon neuron with biologically matched dynamics,” IEEE Trans. Biomed. Eng., vol. 51, no. 2, pp. 342–354, Feb. 2004. [10] T. Yu and G. Cauwenberghs, “Analog VLSI neuromorphic network with programmable membrane channel kinetics,” in Proc. Int. Symp. Circuits Syst., May 2009, pp. 349–352. [11] A. Basu, C. Petre, and P. Hasler, “Bifurcations in a silicon neuron,” in Proc. Int. Symp. Circuits Syst., May 2008, pp. 428–431. [12] E. Farquhar and P. Hasler, “A bio-physically inspired silicon neuron,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 52, no. 3, pp. 477–488, Mar. 2005. [13] T. Takemoto, T. Kohno, and K. Aihara, “MOSFET implementation of class I neurons coupled by gap junctions,” Artif. Life Robot., vol. 10, no. 1, pp. 1–5, Jul. 2006. [14] H. Lecar, “Morris–Lecar model,” Scholarpedia, vol. 2, no. 10, p. 1333, 2007.

[15] A. Basu, R. Robucci, and P. Hasler, “A low-power, compact, adaptive logarithmic transimpedance amplifier operating over seven decades of current,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 49, no. 1, pp. 2167–2177, Oct. 2007. [16] A. Basu, C. Twigg, S. Brink, P. Hasler, C. Petre, S. Ramakrishnan, S. Koziol, and C. Schlottmann, “RASP 2.8: A new generation of floatinggate based field programmable analog array,” in Proc. IEEE Custom Integr. Circuits Conf., Sep. 2008, pp. 213–216. [17] P. Livi and G. Indiveri, “A current-mode conductance-based silicon neuron for address-event neuromorphic systems,” in Proc. Int. Symp. Circuits Syst., May 2009, pp. 2898–2901. [18] F. Folowosele, A. Harrison, A. Cassidy, A. G. Andreou, R. EtienneCummings, S. Mihalas, E. Niebur, and T. J. Hamilton, “A switched capacitor implementation of the generalized linear integrate-and-fire neuron,” in Proc. Int. Symp. Circuits Syst., May 2009, pp. 2149–2152. [19] K. Hynna and K. Boahen, “Neuronal ion-channel dynamics in silicon,” in Proc. Int. Symp. Circuits Syst., May 2006, pp. 3614–3617. [20] A. Bandyopadhyay, G. Serrano, and P. Hasler, “Adaptive algorithm using hot-electron injection for programming analog computational memory elements within 0.2% of accuracy over 3.5 decades,” IEEE J. Solid-State Circuits, vol. 41, no. 9, pp. 2107–2114, Sep. 2006. [21] R. Sarpeshkar, “Efficient Precise Computation With Noisy Components: Extrapolating From an Electronic Cochlea to the Brain,” Ph.D. dissertation, California Inst. Technol., Pasadena, CA, 1997. [22] E. M. Izhikevich, “Which model to use for cortical spiking neurons?,” IEEE Trans. Neural Netw., vol. 15, no. 5, pp. 1063–1070, Sep. 2004. Arindam Basu (S’07) received the B.Tech. and M.Tech degrees in electronics and electrical communication engineering from the Indian Institute of Technology (IIT) Kharagpur, India, in 2004 and 2005, respectively, and the M.S. degree in mathematics and the Ph.D. degree in electrical engineering from the Georgia Institute of Technology, Atlanta, in 2009 and 2010, respectively. He is an Assistant Professor in the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. His research interests include nonlinear dynamics and chaos, modeling neural dynamics, low-power analog IC design, and programmable circuits and devices. Dr. Basu received the JBNSTS award in 2000 and the Prime Minister of India Gold Medal in 2005 from IIT Kharagpur. He is also the corecipient of the Best Student Paper Award in the Ultrasonics Symposium in 2006 and a finalist in the Best Student Paper Contest at ISCAS 2008.

Paul E. Hasler (S’87-M’95-SM’04) received the M.S. and B.S.E. degrees in electrical engineering from Arizona State University, Phoenix, in 1991 and the Ph.D. degree in computation and neural systems from the California Institute of Technology, Pasadena, in 1997. He is currently an Associate Professor with the Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta.

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on July 08,2010 at 08:30:48 UTC from IEEE Xplore. Restrictions apply.

Nullcline-Based Design of a Silicon Neuron

present simulations and measured data from circuits fabricated in 0.35- m CMOS that ... pattern recovery [1]–[3]. ...... Instead, we plan to use FG transistors,.

1MB Sizes 3 Downloads 201 Views

Recommend Documents

Bifurcations in a Silicon Neuron
Theoretical analysis of ... We hope that this analysis not only helps in the design ..... [3] R.L. Calabrese G.N. Patel, G.S. Cymbalyuk and S.P. DeWeerth, “Bi-.

Dynamics and Bifurcations in a Silicon Neuron
department of Electrical and Computer Engineering, Georgia Institute of. Technology ..... 5: Bifurcation in numerical integration: The bifurcation of the theoretical ...

Dynamics and Bifurcations in a Silicon Neuron
obtained from the IEEE by sending an email to [email protected]. EK. ENa. C. C ..... of the theoretical model is observed by continuation using AUTO.

Design, fabrication and testing of a silicon-based air ...
Aug 11, 2006 - of portable electronics have led to the great progress of microsystem ... energy of the reaction of fuel (methanol solution) and oxidant. (oxygen or air) into ..... Micro Mechanical Systems (Florida, USA) pp 600–5. [8] Pichonat T ...

Multiplexer Design in Amorphous Silicon Technology
This thesis presents the design, analysis and implementation of multiplexer circuits ... Circuits are then simulated in Cadence analog design environment using a ...

POGIL 29 Neuron Structure-S.pdf
and features of neurons, which are the primary cells in the nervous system, enable animals to experience. all of the fi ve senses; fi nd food, mates, and shelter; ...

SpikeAnts, a spiking neuron network modelling ... - NIPS Proceedings
observed in many social insect colonies [2, 4, 5, 7], where synchronized patterns of ... chrony [10], order-chaos phase transition [20] or polychronization [11].

Multiphoton-Excited Fluorescence of Silicon ... - Semantic Scholar
May 15, 2017 - to be an enabling technology for minimally invasive in vivo molecular ... within the second near-IR transmission window of tissue .... for use in the IR. The pulse width of the excitation laser beam is determined as a function of wavel

Multiphoton-Excited Fluorescence of Silicon ... - Semantic Scholar
May 15, 2017 - for use in the IR. The pulse width of the excitation laser beam is determined as a function of wavelength using the autocorrelator, substituting a silicon photodiode for the. (Ga,As)P detector ... A spectrum at a location 2 μm distant

The mirror-neuron system: a Bayesian perspective - UCSD Cognitive ...
view and the results of recent studies suggest that the inferior parietal lobule area may be ... 2005; 360:815–836. 16. Wolpert DM, Doya K, Kawato M. A unifying ...

Learning Games using a Single Developmental Neuron
ber of evolutionary generations increases the genotypes de- velop structure that allow the players to play checkers in- creasingly well. We have used an indirect ...

SpikeAnts, a spiking neuron network modelling ... - NIPS Proceedings
ics and computer science, using random Markov fields, cellular automata or ..... analysis of the average foraging effort¯F, versus population size M (top left),.

Reduction of stochastic conductance-based neuron ...
Aug 13, 2011 - Springer Science+Business Media, LLC 2011. Abstract We ... and that this transition accounts for changes in char- ..... small positive parameter) where τm(V) is replaced by ..... tation using a formal calculus software to obtain.

Contributions of intrinsic motor neuron properties to the ...
second half of this paper we discuss recent data showing that the neonatal .... Unpublished data from Kjaerulff and Kiehn. ..... This characteristic has been dem-.

Neuromorphic Silicon Photonics
Nov 5, 2016 - situation is exemplified by radio frequency (RF) process- ... In response, RF photonics has offered respective solutions for tunable RF filters [16, 17], ADC itself [18], and simple processing tasks that can be moved from DSP into the a

Neuromorphic Silicon Photonics
Nov 5, 2016 - existing neural engineering tools can be adapted to silicon photonic information processing systems. A 49-node silicon photonic neural ... faces fundamental physical challenges [4]. Analog optical co-processors have ... photonic integra

Multilayer Silicon Nitride-on-Silicon Integrated Photonic Platform for ...
[email protected], [email protected]. Abstract: A photonic platform with two passive silicon nitride layers atop an active silicon layer is demonstrated.

Silicon Evolution
store," the user can physically create new electronic cir- cuits as easily as writing ..... down a telephone line coded as 10kHz and 1kHz tones: the task here is to ...

Single-neuron responses to intraoral delivery of odor ...
primary olfactory (OC) and gustatory cortex (GC) to intraoral delivery of odor solutions and .... to obtain single neuron spike time stamps (Quiroga et al. 2004).

Journal of Neuroscience Methods One-to-one neuron ...
a turning point with the introduction of the multi-electrode array .... (30 m diameter). Fig. 2. Time lapse observation of cell migration toward CNT islands. Culture is ...