64

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL. 6, NO. 1, FEBRUARY 2012

Small-Signal Neural Models and Their Applications Arindam Basu, Member, IEEE

Abstract—This paper introduces the use of the concept of small-signal analysis, commonly used in circuit design, for understanding neural models. We show that neural models, varying in complexity from Hodgkin–Huxley to integrate and fire have similar small-signal models when their corresponding differential equations are close to the same bifurcation with respect to input current. Three applications of small-signal neural models are shown. First, some of the properties of cortical neurons described by Izhikevich are explained intuitively through small-signal analysis. Second, we use small-signal models for deriving parameters for a simple neural model (such as resonate and fire) from a more complicated but biophysically relevant one like Morris–Lecar. We show similarity in the subthreshold behavior of the simple and complicated model when they are close to a Hopf bifurcation and a saddle-node bifurcation. Hence, this is useful to correctly tune simple neural models for large-scale cortical simulations. Finaly, the biasing regime of a silicon ion channel is derived by comparing its small-signal model with a Hodgkin–Huxley-type model. Index Terms—Bifurcations, neuromorphic system, parameter tuning, silicon ion-channels, small-signal model, spiking neurons.

I. NEURAL MODELS AND SMALL-SIGNAL CIRCUITS HE importance of nonlinear dynamics and bifurcations in the differential equations describing neural activity have long been of interest to scientists [2]–[4] and are still a major topic of research. Recently, Izhikevich reported the nonlinear dynamical explanations for several phenomenon observed in cortical neurons [5]. In this paper, we use the concept of smallsignal analysis, which is a workhorse for the circuit designer, to explain some of those excitability features of nerve membrane models. We show that the small-signal model is markedly different when the neuron is close to a Hopf bifurcation as opposed to a saddle-node bifurcation. We hope that the smallsignal model allows circuit designers to understand the different neural dynamics more intuitively. The basic concept is illustrated in Fig. 1 where a neuron is simplified into a small-signal circuit and a block for thresholding and resetting its state. We also demonstrate the application of small-signal models in determining model parameters of neuronal equations. The Hodgkin–Huxley (H-H) model [6] is typically used by biologists to model detailed kinetics of ion channels. However, using such a complicated model is numerically intractable for largescale neural simulations trying to mimic a human cortex. Also,

T

Manuscript received December 07, 2010; revised March 28, 2011; accepted May 15, 2011. Date of publication July 14, 2011; date of current version January 27, 2012. This paper was recommended by Associate Editor B. Shi. The author is with the VIRTUS, IC Design Centre of Excellence, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798 (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TBCAS.2011.2158314

Fig. 1. Circuit model comprising resistors, capacitors, and inductors can be used to describe the subthreshold excitability of neurons.

finding the correct set of parameters for a desired behavior is very tedious. Hence, computational neuroscientists have developed simpler models, such as integrate and fire, resonate and fire [7], generalized integrate and fire (gif) [8], spike response model [9], etc., which capture some of the essential properties of the more realistic models but are numerically efficient. Finding the correct parameters for the simpler neural model to match its properties with the complicated model is not necessarily an easy task; this requires solving an overdetermined problem since the number of parameters in the complicated model are more than that in the simpler one. We show that small-signal models provide a method for translating parameters across these models. This concept can also be used for tuning silicon neurons such as in [10]. It should be noted that we are not advocating modeling neurons by small-signal linear models for large-scale simulations. Rather, we envision using these techniques for better understanding neuronal dynamics and translating parameters across neuron models. In circuit design, small-signal analysis involves linearizing a nonlinear element around its operating point to generate a circuit comprising linear circuit elements. A similar concept was used earlier in [11, Ch. 10] to intuitively understand the behavior of Hodgkin–Huxley channel models. We generalize that method for arbitrary neural models and apply the same principle to translate parameters across different models. In [12], transformations are applied to the input stimulus to make the subthreshold behavior of integrate and fire neurons and Hodgkin-Huxley neurons similar. Instead of modifying the input, we choose parameters optimally to match the subthreshold behaviors of the two models. Thus, there is no need to know the type of stimulus apriori. Some initial results using this concept were presented in [13]. Here, we expand on the earlier work and give intuitive explanations for some neural excitability phenomenon, provide quantitative measures of the goodness of fit of the spike train response of a simple model using translated parameters from a complex model, and extend the method to silicon neurons. In Section II, we introduce the theory and illustrate its use with an example. In Section III, the small-signal model for a neuron close to saddle-node or Hopf bifurcation is derived

1932-4545/$26.00 © 2011 IEEE

BASU: SMALL-SIGNAL NEURAL MODELS AND THEIR APPLICATIONS

65

and several excitability properties are explained intuitively. Section IV deals with parameter translation from a complicated to a simpler software model while the same is done for a silicon neuron in Section V. Finally, we conclude in the last section. Fig. 2. Small-signal membrane impedance for the Morris–Lecar model.

II. SMALL-SIGNAL NEURAL MODELS A. Theory To describe the concept of small-signal neural models, we start with a general set of differential equations describing a point neuron given by

(1) where represents the membrane voltage, is the normalized current stimulus obtained by dividing the current with the represents a vector of “ ” gating membrane capacitance, is a vector of “ ” parameters. We assume variables and , the aforementioned system has a stable equithat for . Now considering a small current stimulus librium at is incident on the membrane, we can find its small-signal impedance by taking derivatives of (1) to obtain

where is the membrane capacitance, is the maximal is the reversal potential where “ ” conductance, and and corresponds to calcium, potassium, or leak channels. are half activation potentials for calcium and potassium, are the slope factors for the same. respectively, while and To derive the small-signal impedance for the M-L model, we again assume that a small current stimulus is incident on the membrane (as in (2)) to obtain

(5) Thus, we see that the calcium and the leak channels directly contribute conductances to the membrane impedance while the . potassium channel provides an impedance that depends on To obatin the exact impedance, we use the Laplace transform on the equation for the gating variable W to get

(2) (6) where Df and Dg are the Jacobian matrices consisting of partial derivatives for the functions f and g, respectively, evalu. Finally, applying the concept ated at the equilibrium of Laplace transforms to (2), we obtain

where . Combining (5) and (6) we obtain

(3) (7)

where is the identity matrix and “ ” is the Laplace transform variable. B. Example—Morris–Lecar Model To clarify the concept, we shall next apply this principle to the Morris–Lecar (M-L) model [14], a biophysically realistic model of the nerve membrane. The Morris–Lecar model is given by the following equations:

(4)

where . We see that the potassium channel and an inductance with contributes a conductance . Fig. 2 plots this small-signal series resistance model. C. Discussion The aforementioned analysis shows that the time-dependent activating potassium channel in the M-L model results in a branch in the small-signal membrane impedance which has an inductance and a series resistance. It is instructive to consider what occurs in the case of these multiple channels whose differential equations have a form similar to the one considered before (that is, typically the case for H-H type channel models). In that case, each channel will result in an impedance branch in parallel to the membrane capacitance. Each branch will have a series inductance and resistance. The impedances will be positive for negative feedback channels (as in this example) and negative for positive feedback channels (such as the activating

66

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL. 6, NO. 1, FEBRUARY 2012

TABLE I SMALL-SIGNAL IMPEDANCES FOR MORRIS–LECAR MODEL

Fig. 3. M-L model can be tuned to exhibit the (a) Hopf and (b) Saddle-node bifurcations at low input currents when the equilibrium (denoted as “eqbm”) becomes unstable or disappears, giving birth to a limit cycle.

variable for the sodium channel in [6]). In the case of the M-L model, since the positive feedback channel is assumed to be instantaneous, the negative impedance contributed by it is cap. In all of these cases, the matrix is diagonal. tured in Nondiagonal matrices will result if the channel variables are mutually coupled like in the case of inactivation depending on activation [15] in the transient sodium channel. III. NEURAL DYNAMICS In this section, we shall intuitively explain some dynamical properties of nerve membranes using the small-signal impedance model. It has been shown that the neural properties depend on the “bifurcation” that the corresponding differential equation undergoes [2], [5]. A bifurcation refers to the appearance of a qualitative or topological change in the phase portrait of a differential equation as a parameter being varied. The codimension of a bifurcation is the minimum number of parameters that have to be varied for the bifurcation to occur. Bifurcations with a high codimension are rarely observed in experiments since their occurrence depends on several parameters. We shall consider the two commonly observed bifurcations of codimension I—Hopf and Saddle-node. A reason for choosing the M-L model example in the previous section is that it can be tuned to exhibit both bifurcations. Fig. 3 plots the bifurcation diagrams for the two cases obtained by using continuation techniques in AUTO, a freely available software package. The parameter set 100, 50, 70, for Fig. 3(a) is 1.1, .5, 2, 1, 15, 0, 30, gain 5. We use the units of millivolts for voltages, milliseconds for time, pico Amperes for current, nS for conductances, and we assume that the membrane capacitance is 1 pF (this convention is followed in the remainder of this paper). The major difference in the parameters for the case in Fig. 3(b) is that potassium activation is steeper and occurs at a higher 10, 30, 3. membrane voltage quantified by

Fig. 4. Magnitude of the membrane impedance of the M-L model is plotted for the (a) Hopf and (b) Saddle-node bifurcation cases. The parameter values for the two cases are mentioned in the text.

For both these cases, we evaluate the impedances shown in (7). All of these impedances are evaluated with V and W at their equilibrium values. These small-signal impedances are presented in Table I for the two parameter sets mentioned 23.7 and 7.85 for the Hopf and Saddle-node earlier with cases, respectively. Fig. 4(a) and (b) plots the magnitude of the small-signal impedances for the Hopf and Saddle-node cases, respectively. In both cases, the leak and the potassium channel contribute positive conductances while the calcium channel provides a negative conductance due to positive feedback. However, in the saddle-node case, the impedance decreases monotonically with frequency (low-pass) while in the Hopf case, it attains a maximum value at an intermediate frequency (band-pass). This can be readily understood from Fig. 2 and the circuit theoretic concept of quality factor or “ ” of an inductor. The quality factor

BASU: SMALL-SIGNAL NEURAL MODELS AND THEIR APPLICATIONS

67

Fig. 6. Small-signal membrane impedance for (a) the resonate and fire model and (b) the quadratic integrate and fire model.

Fig. 5. Four of the twenty dynamical phenomenon observed in cortical neurons presented in [1]: (J)subthreshold oscillations, (K)resonant excitability, (M)rebound spike, and (R)accommodation. (Electronic version of the figure and reproduction permissions are freely available at www.izhikevich.com).

is the ratio of the impedance of the inductor (L) to the series resistor ; obviously it varies with frequency. If the “ ” is greater than 1 at the frequency where the imaginary part of the impedance of the network in Fig. 2 is zero (or in circuit-theoretic terms, the inductor and capacitor resonate), the circuit exhibits , the sea band-pass-like response. At frequencies where ries impedance is dominated by the resistor. In the terminology used by Izhikevich [5], the high membranes ( dominates in Fig. 2) act as “resonators” while the low ones act as “integrators” ( dominates in Fig. 2). Now, we can intuitively predict some of the dynamical properties of “resonators” by using this circuit model to predict the neuron’s subthreshold behavior. We still consider the neuron to have a fixed voltage threshold and it fires a spike if the membrane voltage crosses this threshold. Thus, we pass the input current through a linear R-L-C filter and apply a threshold nonlinearity to the output of the filter. Fig. 5 (taken from [1]) shows four of these properties which we explain one by one as follows. Subthreshold oscillations: When the neuron is excited by a short pulse of current that is not strong enough to cause it to fire a spike, its membrane voltage exhibits oscillations. This property is obvious for a band-pass filter since its impulse response shows damped oscillations. Resonator: When the neuron is excited by a pair of short pulses., with each being individually not strong enough to cause it to fire a spike, the neuron is more likely to fire when the timing between the input pulses attains an intermediate value. This is a direct consequence of membrane impedance being bandpass; the amplitude of the oscillation of the membrane voltage is maximum when the frequency of the input pulse train coincides with the band of frequencies where its gain is high. If the resulting membrane voltage is higher than the neuron’s spike threshold, it will fire a spike. Rebound spike: When the neuron is subjected to an inhibitory input, it may fire a spike after the release of inhibition. This is again because of the impulse response

of a bandpass system exhibiting damped oscillations and, hence, overshoots the equilibrium irrespective of the sign of the impulse. If the overshoot is large enough to cross the threshold of the neuron, it will fire a spike. Accommodation: When a small but fast ramp of current is injected, the neuron fires a spike. However, if the ramp is slow, a much larger value of current does not elicit a spike. The small but fast ramp has a larger high-frequency content in its spectrum compared to the slower input; hence, it is more likely to elicit a larger membrane voltage displacement, leading to larger chances of eliciting a spike. Thus, we can still explain a lot of neural dynamics by a simple threshold model if we modify the subthreshold impedance model according to the bifurcation that the neuronal equation undergoes. IV. PARAMETER TRANSLATION: SOFTWARE MODEL In this section, we will use the earlier derived concept of small-signal models to obtain parameters for simple neural models from more complex biophysical models so that both of the systems exhibit a similar response to input stimulus. We obtain the parameters by equating the small-signal impedances of both models. We now consider two different simpler models that capture the properties of the differential equations close to the two different bifurcations. We choose the resonate and fire (raf) model [7] and the quadratic integrate and fire [5] (qif) model as the simplest models that capture the properties of the neural models close to a Hopf and a saddle-node bifurcation, respectively. First, we consider the raf model given by

(8) Applying the earlier concepts, we can derive the small-signal impedance for this model as (9) which is shown in Fig. 6(a). The circuit elements in this schematic are exactly the same as the ones in Fig. 2 validating the fact that the raf model is indeed a good approximation to neural models close to a Hopf bifurcation. Next, we consider the qif model given by

for

ms.

(10)

68

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL. 6, NO. 1, FEBRUARY 2012

Fig. 7. Series combination of the inductor and resistor can be converted to a parallel combination so that either impedances are the same at a certain frequency or the eigenvalues of the Jacobian are the same in both cases.

whose small-signal model is given by (11) Thus, the small-signal model for the qif model is a low-pass filter and is similar to that of the Morris–Lecar model close to the saddle-node bifurcation. It should be noted that the leaky integrate and fire model [5] also has a similar small-signal impedance; the qif model, however, provides a better approximation to the spike timing since it models the nonlinearity of the differential equation close to a saddle-node bifurcation. All of these models, however, behave as an integrator and show excitability for inputs that arrive close in time. Some other simple models that have received attention in the neuroscience and neuromorphic engineering community are the Izhikevich model [16] and the gif model proposed by Mihalas and Niebur [8], [17]. These can exhibit much richer dynamics than the raf and the qif models but have more parameters as well. We do not consider them in detail here, but derive a small-signal equivalent model for them in the Appendix.

A. Subthreshold Behavior In this section, we shall derive the parameters of the raf and qif models so that their subthreshold dynamics match that of the appropriately biased Morris–Lecar model. In the first case, we see that the original 11 parameters of the M–L model have been reduced to three effective parameters for the subthreshold behavior: , , and while the raf model has two parameters and . But it can be seen by comparing Figs. 2 and 6(a) that a direct translation of parameters by matching coefficients is not possible since the raf model has two free parameters while the small-signal model for M–L needs 3. Here, we can use our knowledge of circuit theory and convert the series inductor-resistor combination into a parallel combination, as shown in Fig. 7, which has a similar impedance and quality close to the resonance frequency. Now, the two parfactor allel resistors can be combined into one, reducing the number of independent components to 2. The equations governing this transformation [18] are (12) where and are the parallel and series impedances, respectively. Another alternate way to make this transformation is to equate the eigenvalues of the Jacobian for the series and parallel models

Fig. 8. Response of a series resistance and a parallel resistance model to (a) a periodic train of impulses and (b) a single impulse. The parallel impedance values are derived by either equating eigenvalues or impedances. The two responses match better for the periodic input case when the parallel model is derived by equating impedances.

since the eigenvalues determine asymptotic stability of the equilibrium. This leads to a slightly different set of values given by

(13) In Section IV-B, we shall show the results of spike train predictions for both cases. But it is interesting to note that equating the impedances for the two models at some frequency makes particular solutions of the two systems the same when they are driven with a sinusoid at that frequency. Since the neuron will operate under forcing current stimulus and is most likely to fire a spike when the input is close to the resonant frequency, it is reasonable to equate the impedances at that frequency even if the input stimulus is not sinusoidal. To validate this hypothesis, Fig. 8(a) and (b) plots the response of a series resistance network and its equivalent parallel resistance circuit for a periodic train of current impulses as input or a single impulse input, respectively. The values of the parallel network are derived by either equating impedance at the resonant frequency or the eigenvalues

BASU: SMALL-SIGNAL NEURAL MODELS AND THEIR APPLICATIONS

69

of the Jacobian. Differential equations for both cases were numerically simulated in MATLAB using a forward Euler method 0.1, and parameter values 1, with time step 1, 5, and 0. Indeed, it can be seen in Fig. 8(a) that the response of the parallel network is closer to the original series network when it is derived by equating the impedances at the resonant frequency. However, Fig. 8(b) shows that the other parallel network is able to track the solution of the series network better over a time period that is longer than the resonant time period. We can apply these two methods to the M-L small-signal model shown in Fig. 4 to obtain the resulting parallel network , , and 1, regiven by replacing , , and in (12) by spectively. Next, by applying these transformations to the raf model, we obtain Resonant Impedance Match Eigenvalue Match

(14)

Now, we can compare the two resulting parallel networks to obtain the parameters of the raf model as follows: Resonant Impedance Match

Eigenvalue Match

(15) Next, we derive the parameters for the saddle-node case. This situation is simpler and we can directly compare the two smallsignal impedance models to obtain (16) is approximated by subjecting the neuron to inwhere creasing current pulses and noting the largest membrane voltage excursion without a spike being fired. Determining the threshold and other spike generation-related parameters will be discussed in more detail in Section IV-B. We now compare the subthreshold responses of the two models when it is subject to a train of random synaptic inputs. The EPSC is approximated by a current pulse of width 2 ms, whose amplitude follows a uniform random distribution between 0 and 2 pA. Numerical integration is performed in MATLAB by using a forward Euler method with time step 0.5 ms. Fig. 9 plots the comparison between M-L and (a) raf and (b) qif models. We can see that the subthreshold responses are well matched in both cases. In Section IV-B, we shall see how the models behave for the above threshold inputs and how accurately we can capture the output spike train for both models.

Fig. 9. Comparison of the subthreshold dynamics of the M-L model with (a) raf and (b) qif models. The synaptic inputs are approximated by 2-ms-wide current pulses whose amplitudes follow a uniform distribution between 0 and 2 pA. The raf and qif waveforms are shifted so that their equilibrium membrane voltages match that of the M-L model, enabling easier comparison.

B. Above Threshold Behavior The major use of the simple neuron models is utilizing these for large-scale network simulations. For many simulations, the timing of the spikes is considered an important metric which codes information or is responsible for learning algorithms, such as Spike-timing-dependent plasticity [19], [20]. Hence, we next explore the ability of the simpler models, tuned by using smallsignal analysis, in predicting spike timing. First, we need to estimate the threshold of the simple neuron for firing spikes. To accomplish that, we first subject the M-L neuron model to a single, 2-ms-wide synaptic current pulse and increase the amplitude of the pulse until the neuron spikes. This process is shown in Fig. 10(a) where we estimate the synaptic current threshold to be between 3.27 and 3.28 pA. Next, the same set of input current pulses is injected on the raf neuron as shown in Fig. 10(b). The figure in the inset shows a zoomed plot of the perturbed state of the neuron after the synaptic input is chosen as the black has been applied. The voltage line in the inset figure so that the 3.28-pA synaptic current is an above threshold input while the 3.27-pA input is not. The

70

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL. 6, NO. 1, FEBRUARY 2012

Fig. 11. Reset voltage and reset time of the (a) qif model and the (b) raf model are chosen to match that of the biophysically realistic one. The dashed line in both figures denotes the simple model.

Fig. 10. (a) Input current pulses of width 2 ms and amplitude 3.23, 3.27, and 3.28 pA are injected on the membrane of the M-L model to determine its threshold to synaptic current input being 3.28 pA. (b) The same current pulses are injected on the membrane of the raf model and V is set to make its threshold to synaptic current pulses also 3.28 pA.

same technique is applied to find the threshold for the qif neuron as well. Of course, this is only an estimate of the “threshold” since we are approximating a manifold by a straight line. (More details on how the stable manifold of a saddle node may act as a threshold in neuron models can be found in [5].) and are next chosen to match the The parameters postspike dynamics of the two models. For the qif model, we first measure the difference in voltage between the minimum after a spike and the equilibrium potential in the value of is set equal to this value and is chosen M-L model. so that both models take the same time to reach steady state after firing a spike. An example is shown for the qif model in and are chosen to match the postspike Fig. 11(a) where voltage trajectory in the two models. in the For the raf model, however, we cannot choose same way since a very negative reset voltage may make the neuron fire repeatedly. The reason for this is that the threshold manifold for the raf model wraps around the equilibrium [7] and if the reset point is outside this curve, the neuron will again reach the threshold after being reset. This issue is not present in the qif model since it is 1-D and the threshold is just a point. Hence, we where “ ” is a margin that enchoose sures that the post reset trajectory is away from the boundary of the threshold manifold. A range of values has been found to work for this parameter. In the subsequent simulations, we

chose it to be 0.1. determines the phase at which the subthreshold oscillations start after the spike. Hence, for every value , a value of exists for which the phase of this oscillaof tion can be matched approximately with that of the M-L model has to be chosen so that the reset point after the spike. Also, is inside the threshold manifold. Fig. 11(b) shows an example of choosing the reset parameters for the raf neuron so that its response matches the M-L neuron. Fig. 12 compares the spike responses for the Morris–Lecar model with (a) raf and (b) qif models when they are subjected to a stream of random synaptic inputs with EPSC amplitudes drawn from a random distribution varying from 0 to 3.5 pA. We see that there is a close correspondence in the generated spikes in both cases. However, there are some errors which are related to the reset mechanism and threshold estimation (threshold voltage actually corresponds to a manifold, that is, it depends on the other variables). To quantify the goodness of the matching of spike times between two spike trains, we use a measure introduced by Jolivet in [21], which is given by (17) , , and are the number of matching spikes, where total number of spikes in the original spike train, and total number of spikes in the predicted spike train. is the timing precision of spike matching (i.e., a spike in the predicted train corresponds to another spike in the original spike train if they occur within a time interval of each other). is the average rate of spike occurrence in the original spike train. The measure is normalized to have a value of 1 for perfect match and 0 for no denotes the expected matches. It should be noted that number of matched spikes for any two poisson spike trains with a mean rate and matching accuracy . Thus, by subtracting , we ensure that only valid spike this number from matches are being considered and the measure also becomes and matching window relatively robust to spike rate variations. Following [22], we choose to be a multiple of the width of a spike; a range of values gives very similar results. In to be equal to four times our simulations, we have chosen the width of a spike. Fig. 13(a) and (b) plots for the cases of raf and qif, respectively. In these plots, the axis shows the maximum amof the EPSC. In other words, in each experiment, plitude

BASU: SMALL-SIGNAL NEURAL MODELS AND THEIR APPLICATIONS

71

Fig. 12. Comparison of the above threshold dynamics of the ML model with (a) raf and (b) qif models. The synaptic inputs are approximated by 2-ms-wide current pulses whose amplitudes follow a uniform distribution between 0 and 3.5 pA. The waveforms are shifted and the spikes from the Morris–Lecar model are clipped at 0 mV.

the EPSC amplitude is a random number between 0 and . A vertical line is drawn at the EPSC amplitude which is the current threshold of the neuron. For each data point, an input current waveform was injected for 1 min and the value of was computed. The curve is not very smooth for low values of because the neuron fires a small number of spikes when the input is very small, leading to crude approximations of the goodness of fit. For the raf case, we plot the values for both cases when the parameters are obtained by matching resonant impedances or eigenvalues. Though both curves are very close, the resonant impedance method gives slightly better results for low-input amplitudes when the inputs do need to have a precise timing pattern to excite the neuron. It should be noted that we do not claim that this is the best way to optimize parameters for predicting spike trains. In fact, there have been other cases where many optimization algorithms [23], [24] are used to fine-tune model parameters for spike timing predictions. Moreover, the two models raf and qif are not intended for quantitative predictions; they have been developed to capture neuron dynamics qualitatively. Given these caveats, we found it interesting to note that just the linear small-signal model was able to attain values as high as 0.75 when all of the inputs were below threshold. This is only possible if the interaction of the subthreshold inputs with the neuron membrane is modeled fairly well. This value increases more when above threshold inputs are also included since the prediction is correct almost

Fig. 13. Goodness of prediction 0 is plotted for the (a)raf and (b)qif models against increasing maximum EPSC amplitude. The vertical line denotes the threshold of EPSC for spike generation.

every time the input is strong enough to make the neuron fire by itself. If an optimization program is run for finding parameters, we expect that these small-signal-based parameters may be used as a good initial guess to increase convergence speed. Also, this method gives, at most, two constraining equations for every gating variable. If the chosen simpler model has more free parameters, other constraining equations need to be used to find the final parameter values. V. PARAMETER TRANSLATION: SILICON CHANNEL In this section, we shall use small-signal models to determine the biasing regime for neuromorphic silicon neuron designs. Neuromorphic engineers have designed several analog circuits [25]–[29] mimicking the behavior of neural differential equations. For using these systems to replace digital simulations, a methodology is needed to determine the neuron circuit’s biases according to the theoretical model’s parameters. Voltage-clamp-based methods [30], [31] were used earlier to find circuit biases; the small -ignal model provides an alternative. We shall obtain bias voltages for the transient sodium channel presented in [28] that models H-H such as activation and inactivation.

72

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL. 6, NO. 1, FEBRUARY 2012

Fig. 14. Small-signal model for the H-H sodium channel model. The inactivation equation results in the positive impedance branch while the activation equation results in the negative impedance branch.

Let us first obtain the small-signal model for the H-H sodium channel equations given by

(18)

Fig. 15. Circuit implementing H-H, such as sodium channel dynamics, as presented in [28].

at initially goes down, increasing the positive feedback depolarizing current through with a time constant guided . This is equivalent to the activation equation in the by H-H model. However, current through M3 acts as inactivation up to its initial equilibrium and reduces and slowly pulls . This occurs on a time scale the channel current through . guided by For this circuit, we can obtain the small-signal model by directly replacing the transistors with their equivalent small-signal models. Ignoring parasitic capacitances related to the transistors and assuming infinite early voltage, we obtain

(19) where the variables , , and represent the fraction of activated channels, fraction of inactivated channels, and membrane voltage, respectively. The parameters in the simulation 60, 2400, 40, 15, have values 62, 7, 1, and 9, where voltages are in millivolts, conductances are in nS, and time is in milliseconds as mentioned earlier in this paper. It should be noted that this is a slightly simplified form of the original H-H model since the variation of time constant with membrane voltage is not modeled. is considered to be . Using the The steady-state voltage theory developed earlier, the small-signal model for this set of equations can be derived and is shown in Fig. 14. The values of the small-signal impedances are given by

(20) It can be seen that the positive feedback activation variable provides a negative impedance branch while the negative feedback inactivation variable provides a positive impedance. The negative impedance branch can be converted to a positive impedance branch shunted by negative conductance as shown in Fig. 14. Next, we derive the small-signal model for the sodium channel circuit shown in Fig. 15. For the sake of completeness, we briefly describe the operation of the circuit while we refer the reader to [28] and [32] for details. The transistor represents the sodium channel, while the rest of the circuit is a bandpass amplifier for generating the gating potential for . For a depolarizing step current input at , the voltage

(21) and are the values of sodium channel current where and gate voltage of M1 at rest. This leads to the small-signal equivalent circuit drawn in Fig. 16. Compared with Fig. 14, we can immediately see some similarities: the positive feedback activation provides a negative impedance branch while the negative feedback inactivation provides a positive impedance. Also, the in each branch is determined by the respective time constants. However, there are some differences as well. The branch and is missing in the H-H equivalent circuit. with This is not a major problem though, since at low frequencies, this branch is capacitive and can be lumped with the membrane capacitance while at high frequencies, this provides a fixed conductance. The other major difference is that the positive and negative impedance branches in the silicon model have the same conductance. This is a result of the fact that the silicon model has exactly zero gain at dc (capacitively coupled amplifier) while the H-H model has a nonzero gain at dc. To better explain these points, the small-signal admittances of the silicon and the H-H model are plotted against frequency in

BASU: SMALL-SIGNAL NEURAL MODELS AND THEIR APPLICATIONS

73

Fig. 16. Small-signal model for the silicon sodium channel. Similar to the H-H case, the inactivation equation results in the positive impedance branch while the activation equation results in the negative impedance branch.

Fig. 18. Both the silicon and H-H numerical model are subjected to (a) voltage steps of different amplitudes and (b) voltage ramps of fixed amplitude but different slopes. The outputs are better matched at shorter time scales than longer ones indicating the difference in low-frequency gain. Fig. 17. Small-signal admittance for the sodium channel in the H-H model and the silicon version. The curve for the ideal circuit is obtained by replacing the transistors with their small-signal models without parasitic capacitances.

Fig. 17. These plots are generated by SPICE simulations using EKV transistor models for a TSMC 0.35- m CMOS process. The parameters for the silicon circuit were obtained by first and and then choosing equating which matches the shunt conductance in the two models. Other are also possible to match it with , choices of or an average of these three. The curve marked “ideal circuit” is obtained by replacing the transistors with their small-signal model without parasitic capacitances. The difference in low-frequency gain can be readily observed since the admittance of the silicon model is entirely capacitive at low frequencies unlike the H-H model which has a conductance component. However, the frequency variation of the admittance where the channel has its highest gain is well matched in both models. A small difference between the actual silicon circuit and the ideal circuit can be seen at high frequencies where the admittance of the actual circuit again becomes capacitive due to the transistor parasitics. However, this occurs at a frequency that is much higher than the range we are interested in. Using these parameters, we can now compare the behavior of the H-H model and the silicon circuit when subjected to voltage steps and ramps. These are necessary to characterize the channel behavior for usage in dynamic clamp experiments. The H-H model was numerically integrated in MATLAB with a time step

of 100 s. First, voltage steps of 0.1, 0.3, and 0.5 mV were applied to both models, and the results are plotted in Fig. 18(a). The initial activation part is well matched for both models; however, the difference in dc behavior is obvious since the final current values for the two are different. Next the input voltage was ramped by 0.5 mV in 1, 8.6, and 16.2 ms and the outputs are plotted in Fig. 18(b). Again, the difference of the silicon and the H-H model are more on longer time scales. The point of this section is not to argue which of these models is better matched to biology; rather we just use small-signal models to analyze the differences between the two and provide a method for matching the two responses as much as possible. The dc gain of the H-H model is less if larger steps are considered which makes the final inactivation value close to zero. For large steps, the parameters of the silicon model can be obtained by considering a small-signal model that is an average of the ones obtained at the initial and final values of the step. VI. CONCLUSION In this paper, we have developed a method for using small-signal analysis, a tool well-known to circuit designers for studying neural dynamics. We presented a general framework for transforming differential equations representing neuronal models to a circuit comprising resistors, capacitors, and inductors. This model has been used to explain some of the complex phenomenon associated with “resonator”-type neurons [5]. The

74

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL. 6, NO. 1, FEBRUARY 2012

Fig. 19. Small-signal circuit for the Izhikevich model.

Fig. 20. Small-signal circuit for the gif model.

use of the small-signal circuit in determining and translating parameters of different neural models is also explored. The parameters of two simple models are derived from a biophysically inspired Morris–Lecar model and the performance of the two cases is demonstrated. We have also used the same method to derive bias conditions of a silicon channel model from its Hodgkin–Huxley counterpart. We hope that this small-signal neural circuit model allows circuit designers to easily and intuitively understand the dynamics associated with neural differential equations.

ical behavior of the model comes not from the differential equations, which are linear, but from the complex update rules. The model is described by the following equations:

APPENDIX A SMALL-SIGNAL REPRESENTATION OF THE IZHIKEVICH MODEL The simple neuronal model proposed by Izhikevich in [16] can behave as an integrator or a resonator. It can be obtained by adding another equation to the qif neuron and incorporating spike-triggered adaptation. The model is described by the following equations:

(22) where is the membrane potential; is the injected current; , , and are the parameters of the model and 30 mV is the peak of the generated spike. Let us denote its equilibrium by . Applying the earlier theory, we can derive its smallsignal equation to be

(24) is the membrane voltage relative to the resting where , is the membrane capacitance, i is the potential applied current, is the spiking threshold relative to resting threshold , and , , , , , and are free parameters of the model. To derive a small-signal model, we first note that the currents are activated only by spike events and do not govern the intrinsic subthreshold behavior. Hence, they can be ignored for the 1 small-signal model derivation. Also, we assume in keeping with our earlier convention. Next, it should be noted that when we used a small-signal circuit to represent the membrane and applied input current to it, the resulting membrane voltage was compared with a fixed threshold. In this case, a fixed . threshold exists as well; however, it is applied to So if we want to use a small-signal circuit to represent the entire dynamics, we can rewrite the earlier equations in terms of to obtain

(23) The corresponding circuit is drawn in Fig. 19 and is very similar to the M-L model. However, note that one of the conductances is dependent on the equilibrium voltage which, in turn, is dependent on the parameter b. This is very different from the raf and qif models where the equilibrium was at the origin, independent of parameters. Also, the threshold is implicitly set as the earlier by these equations instead of an explicit models. Hence, parameters related to spike generation are coupled with subthreshold behavior in this model. A more general set of equation where the coefficients of the quadratic equation are free parameters will be more useful for spike train predictions since then the subthreshold behavior and the threshold can be set independently.

(25) Now, we can apply our earlier theory to obtain the smallsignal impedance equation as (26) The corresponding circuit is shown in Fig. 20. It should be noted that this model cannot intrinsically behave as a resonator [8]. However, frequency preference can be exhibited by a combination of spike-induced currents. Thus, this model represents a case where complicated update rules are responsible for important dynamics rather than the subthreshold neuronal properties.

APPENDIX B SMALL-SIGNAL REPRESENTATION OF THE GIF NEURON

ACKNOWLEDGMENT

The gif model proposed in [8] uses a variable threshold to model the effect of voltage-dependent currents. The rich dynam-

The author would like to thank Prof. P. Hasler for his helpful comments and suggestions.

BASU: SMALL-SIGNAL NEURAL MODELS AND THEIR APPLICATIONS

REFERENCES [1] E. M. Izhikevich, “Which model to use for cortical spiking neurons?,” IEEE Trans. Neural Netw., vol. 15, no. 5, pp. 1063–1070, Sep. 2004. [2] J. Rinzel and G. B. Ermentrout, Analysis of Neural Excitability and Oscillations, C. Koch and I. Segev, Eds. Cambridge, MA: MIT Press, 1989. [3] R. Fitzhugh, “Impulses and physiological states in theoretical models of nerve membrane,” Biophys. J., vol. 1, no. 6, pp. 445–466, Jul. 1961. [4] R. Fitzhugh, “Thresholds and plateaus in the Hodgkin-Huxley nerve equations,” J. Gen. Physiol., vol. 43, pp. 867–896, May 1960. [5] E. M. Izhikevich, Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting. Cambridge, MA: MIT Press, 2007. [6] A. Hodgkin and A. Huxley, “A quantitative description of membrane current and its application to conduction and escitation in nerve,” J. Physiol., vol. 117, pp. 500–544, 1952. [7] E. M. Izhikevich, “Resonate-and-fire neurons,” Neural Netw., vol. 14, pp. 883–894, 2001. [8] S. Mihalas and E. Niebur, “A generalized linear integrate-and-fire neural model produces diverse spiking behaviors,” Neural Comput., vol. 21, pp. 704–718, 2009. [9] W. Kistler, W. Gerstner, and J. L. van Hemmen, “Reduction of the Hodgkin-Huxley equations to a single-variable threshold model,” Neural Comput., vol. 9, pp. 1015–1045, 1997. [10] A. Basu, C. Petre, and P. Hasler, “Bifurcations in a silicon neuron,” in Proc. Int. Symp. Circuits Syst., May 2008, pp. 428–431. [11] C. Koch, Biophysics of Computation: Information Processing in Single Neurons. Oxford, U.K.: Oxford Univ. Press, 2004. [12] D. I. Standage and T. P. Trappenberg, “Differences in the subthreshold dynamics of leaky integrate-and-fire and Hodgkin-Huxley neuron models,” in Proc. Int. Joint Conf. Neural Netw., 2005, pp. 396–399. [13] A. Basu and P. E. Hasler, “Small signal neural models and its application to determining model parameters,” in Proc. IEEE Biomed. Circuits Syst. Conf., Paphos, Cyprus, Nov. 2010, pp. 174–177. [14] H. Lecar, “Morris-Lecar model,” Scholarpedia, vol. 2, no. 10, pp. 1333–1333, 2007. [15] E. M. Peganov, E. N. Timin, and B. I. Khodorov, “The link between sodium activation and inactivation,” Bull. Experiment. Biol. Med., vol. 76, no. 4, pp. 1127–1131, 1973. [16] E. M. Izhikevich, “Simple model of spiking neurons,” IEEE Trans. Neural Netw., vol. 14, no. 6, pp. 1569–1572, Nov. 2003. [17] A. Van Schaik, C. Jin, A. McEwan, T. J. Hamilton, S. Mihalas, and E. Niebur, “A log-domain implementation of the Mihalas-Niebur neuron model,” in Proc. Int. Symp. Circuits Syst., May 2010, pp. 4249–4252. [18] T. H. Lee, The Design of CMOS Radio-Frequency Integrated Circuits. Cambridge, U.K.: Cambridge Univ. Press, 2004. [19] H. Markram, J. Lubke, M. Frotscher, and B. Sakmann, “Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs,” Science, vol. 275, pp. 213–215, 1997. [20] G. Q. Bi and M. M. Poo, “Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type,” J. Neurosci., vol. 18, pp. 10464–10472, 1998. [21] R. Jolivet, T. Lewis, and W. Gerstner, “Generalized integrate-and-fire models of neuronal activity approximate spike trains of a detailed model to a high degre of accuracy,” J. Neurophysiol., vol. 92, pp. 959–976, 2004.

75

[22] R. Jolivet, “Effective minimal threshold models of neuronal activity,” Ph.D. dissertation, Ecole Polytechnique Federale de Lausanne (EPFL), Lausanne, Switzerland, 2005. [23] E. Lange and M. Hasler, “Predicting single spikes and spike patterns with the Hindmarsh-Rose model,” Biol. Cybern., vol. 99, pp. 349–360, 2008. [24] M. Pospischil, M. Toledo-Rodriguez, C. Monier, Z. Piwkowska, T. Bal, Y. Fregnac, H. Markram, and A. Dexteshe, “Minimal Hodgkin-Huxley type models for different classes of cortical and thalamic neurons,” Biol. Cybern., vol. 99, pp. 427–441, 2008. [25] G. Indiveri, E. Chicca, and R. Douglas, “A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity,” IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 211–221, Jan. 2006. [26] L. Alvado, J. Tomas, S. R.-L. Masson, and V. Douence, “Design of an analogue ASIC using subthreshold CMOS transistors to model biological neurons,” in Proc. IEEE Custom Integr. Circuits Conf., May 2001, pp. 97–100. [27] J. Arthur and K. Boahen, “Synchrony in silicon: The gamma rhythm,” IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1815–1825, Nov. 2007. [28] E. Farquhar and P. Hasler, “A bio-physically inspired silicon neuron,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 52, no. 3, pp. 477–488, Mar. 2005. [29] A. Basu and P. Hasler, “Nullcline based design of a silicon neuron,” IEEE Trans. Circuits Syst I, Reg. Papers, vol. 57, no. 11, pp. 2938–2947, Nov. 2010. [30] S. Saighi, Y. Bornat, J. Tomas, and S. Renaud, “Neuromimetic ICs and system for parameters extraction in biological neuron models,” in Proc. Int. Symp. Circuits Syst., Kos, Greece, May 2006, pp. 4207–4210. [31] S. Saighi, L. Buhry, Y. Bornat, G. N’Kaoua, J. Tomas, and S. Renaud, “Adjusting the neurons models in neuromimetic ICs using the voltageclamp technique,” in Proc. Int. Symp. Circuits Syst., Seattle, WA, May 2008, pp. 1564–1567. [32] A. Basu, C. Petre, and P. Hasler, “Dynamics and bifurcations in a silicon neuron,” IEEE Trans. Biomed. Circuits Syst., vol. 4, no. 5, pp. 320–328, Oct. 2010. Arindam Basu (M’10) received the B.Tech and M.Tech degrees in electronics and electrical communication engineering from the Indian Institute of Technology, Kharagpur, in 2005, and the M.Sc. degree in mathematics and the Ph.D. degree in electrical engineering from the Georgia Institute of Technology, Atlanta, in 2009 and 2010, respectively. Currently, he is an Assistant Professor in the School of Electrical and Electronics Engineering, Nanyang Technological University, Singapore. His research interests include nonlinear dynamics and chaos, modeling neural dynamics, low-power analog integrated-circuit design, as well as programmable circuits and devices. Mr. Basu received the JBNSTS Award in 2000 and the Prime Minister of India Gold Medal in 2005 from the Indian Institute of Technology, Kharagpur. He received the best student paper award, Ultrasonics Symposium, 2006; best live demonstration, ISCAS 2010; and a finalist position in the best student paper contest at ISCAS 2008.

Small-Signal Neural Models and Their Applications - IEEE Xplore

small-signal analysis, commonly used in circuit design, for un- derstanding neural ... Three applications of small-signal neural models are shown. First, some of ...

1MB Sizes 28 Downloads 338 Views

Recommend Documents

Marginal Unit Generation Sensitivity and its Applications ... - IEEE Xplore
marginal unit generation sensitivity (MUGS) may be calculated based on ... times it is not advisable to apply this local information to predict generations at ...

Challenges for Perceptual Computer Applications and ... - IEEE Xplore
Jul 12, 2012 - I. Introduction. The phrase Computing With Words (CWW), originated by Zadeh in 1996 [31], equates fuzzy logic to it (see. Box 1). Oh, if it were ...

Neural Network Based Macromodels for High Level ... - IEEE Xplore
A simple Back Propagation (BP) algorithm is employed to train a feed-forward neural network with the available data set to find out the weights and biases of the interconnecting layers, and subsequently the neural network is used as a model to determ

IEEE Photonics Technology - IEEE Xplore
Abstract—Due to the high beam divergence of standard laser diodes (LDs), these are not suitable for wavelength-selective feed- back without extra optical ...

Impact of Practical Models on Power Aware Broadcast ... - IEEE Xplore
The existing power aware broadcast protocols for wireless ad hoc and sensor networks assume the impractical model where two nodes can communicate if and only if they exist within their transmission radius. In this paper, we consider practical models

wright layout - IEEE Xplore
tive specifications for voice over asynchronous transfer mode (VoATM) [2], voice over IP. (VoIP), and voice over frame relay (VoFR) [3]. Much has been written ...

Device Ensembles - IEEE Xplore
Dec 2, 2004 - time, the computer and consumer electronics indus- tries are defining ... tered on data synchronization between desktops and personal digital ...

wright layout - IEEE Xplore
ACCEPTED FROM OPEN CALL. INTRODUCTION. Two trends motivate this article: first, the growth of telecommunications industry interest in the implementation ...

Evolutionary Computation, IEEE Transactions on - IEEE Xplore
search strategy to a great number of habitats and prey distributions. We propose to synthesize a similar search strategy for the massively multimodal problems of ...

I iJl! - IEEE Xplore
Email: [email protected]. Abstract: A ... consumptions are 8.3mA and 1.lmA for WCDMA mode .... 8.3mA from a 1.5V supply under WCDMA mode and.

Gigabit DSL - IEEE Xplore
(DSL) technology based on MIMO transmission methods finds that symmetric data rates of more than 1 Gbps are achievable over four twisted pairs (category 3) ...

Maximizing user utility in video streaming applications - IEEE Xplore
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 2, FEBRUARY 2003. 141. Maximizing User Utility in Video.

IEEE CIS Social Media - IEEE Xplore
Feb 2, 2012 - interact (e.g., talk with microphones/ headsets, listen to presentations, ask questions, etc.) with other avatars virtu- ally located in the same ...

Grammatical evolution - Evolutionary Computation, IEEE ... - IEEE Xplore
definition are used in a genotype-to-phenotype mapping process to a program. ... evolutionary process on the actual programs, but rather on vari- able-length ...

SITAR - IEEE Xplore
SITAR: A Scalable Intrusion-Tolerant Architecture for Distributed Services. ∗. Feiyi Wang, Frank Jou. Advanced Network Research Group. MCNC. Research Triangle Park, NC. Email: {fwang2,jou}@mcnc.org. Fengmin Gong. Intrusion Detection Technology Divi

striegel layout - IEEE Xplore
tant events can occur: group dynamics, network dynamics ... network topology due to link/node failures/addi- ... article we examine various issues and solutions.

Digital Fabrication - IEEE Xplore
we use on a daily basis are created by professional design- ers, mass-produced at factories, and then transported, through a complex distribution network, to ...

Iv~~~~~~~~W - IEEE Xplore
P. Arena, L. Fortuna, G. Vagliasindi. DIEES - Dipartimento di Ingegneria Elettrica, Elettronica e dei Sistemi. Facolta di Ingegneria - Universita degli Studi di Catania. Viale A. Doria, 6. 95125 Catania, Italy [email protected]. ABSTRACT. The no

Device Ensembles - IEEE Xplore
Dec 2, 2004 - Device. Ensembles. Notebook computers, cell phones, PDAs, digital cameras, music players, handheld games, set-top boxes, camcorders, and.

Fountain codes - IEEE Xplore
7 Richardson, T., Shokrollahi, M.A., and Urbanke, R.: 'Design of capacity-approaching irregular low-density parity check codes', IEEE. Trans. Inf. Theory, 2001 ...

Multipath Matching Pursuit - IEEE Xplore
Abstract—In this paper, we propose an algorithm referred to as multipath matching pursuit (MMP) that investigates multiple promising candidates to recover ...

Pseudosaturation and Negative Differential ... - IEEE Xplore
Feb 20, 2013 - IEEE TRANSACTIONS ON ELECTRON DEVICES, VOL. 60, NO. 3, MARCH 2013. 985. Pseudosaturation and Negative Differential.