Text

An efficient neuronal representation of incoming sensory information should take advantage of the regularity and scale invariance of stimulus features in the natural world. In the case of vision, this regularity is reflected in the typical probabilities of encountering particular visual contrasts, spatial orientations, or colors (1). Given these probabilities, an optimized neural code would eliminate any redundancy, whilst devoting increased representation to commonly encountered features.

At the level of a single spiking neuron, information about a potentially large range of stimuli is compressed into a finite range of firing rates, since the maximum firing rate of a neuron is limited. Maximizing the information transmission through a single neuron in the presence of uniform, additive noise has an intuitive interpretation: the most efficient representation of the input uses every firing rate with equal probability. An analogous principle for non-spiking neurons has been tested experimentally by Laughlin (2), who matched the statistics of naturally occurring visual contrasts to the response amplitudes of the blowfly's large monopolar cell.

From a theoretical perspective, the central question is whether a neuron can ``learn" the optimal representation for natural stimuli through experience. During neuronal development, the nature and frequency of incoming stimuli are known to change both the anatomical structure of neurons and the distribution of ionic conductances throughout the cell (3). We seek a guiding principle that governs the developmental timecourse of the $\text{Na}^{+}$, $\text{Ca}^{2+}$ and $\text{K}^+$ conductances in the somatic and dendritic membrane by asking how a neuron would set its conductances to transmit as much information as possible. Spiking neurons must associate a range of different inputs to a set of distinct responses--a more difficult task than keeping the firing rate or excitatory postsynaptic potential (EPSP) amplitude constant under changing conditions, two tasks for which learning rules that change the voltage-dependent conductances have recently been proposed (4; 5). Learning the proper representation of stimulus information goes beyond simply correlating input and output; an alternative to the classic postulate of Hebb (6), in which synaptic learning in networks is a consequence of correlated activity between pre- and postsynaptic neurons, is required for such learning in a single neuron.

To explore the feasibility of learning rules for information maximization, a simplified model of a neuron consisting of two electrotonic compartments, illustrated in Fig. 1, was constructed. The soma (or cell body) contains the classic Hodgkin-Huxley sodium and delayed rectifier potassium conductances, with the addition of a transient potassium ``A-"current and a spike-dependent ``adaptation" current that models the slowing of the spike rate in response to sustained current injection. The soma is coupled through an effective conductance G to the dendritic compartment, which contains a synaptic input conductance and three adjustable calcium and three adjustable potassium conductances.
 

  Figure 1  

The model neuron contains two compartments to represent the cell's soma and dendrites. To maximize the information transfer, the parameters for three calcium and three potassium voltage-dependent conductances in the dendritic compartment are iteratively adjusted, while the somatic conductances responsible for the cell's spiking behavior are held fixed.
 

The dynamics of this model are given by Hodgkin-Huxley-like equations that govern the membrane potential and a set of activation and inactivation variables, m_i and h_i, respectively. In each compartment of the neuron, the voltage V evolves as

$$C {d V \over dt} = \sum_i g_i \, m_i^{p_i}\, h_i^{q_i} \, (E_i - V),$$ (1)

where C is the membrane capacitance, g_i is the (peak) value of the i-th conductance, p_i and q_i are integers, and E_i are the ion-specific reversal potentials. The variables h_i and m_i obey first order kinetics of the type $d m /dt = \left( m_{\infty}(V) - m \right)/\tau(V),$where $m_{\infty}(V)$ denotes the steady state activation when the voltage is clamped to V and $\tau(V)$ is a voltage-dependent time constant (7).

Hodgkin-Huxley models can exhibit complex behaviors on several timescales, such as firing patterns consisting of ``bursts"--sequences of multiple spikes interspersed with periods of silence. We will, however, focus on models of regularly spiking cells that adapt to a sustained stimulus by spiking periodically. To quantify how much information about a continuous stimulus variable x the time-averaged firing rate f of a regularly spiking neuron carries, we use a lower bound (8) on the mutual information I(f; x) between the stimulus x and the firing rate f: 

$$I_{\text{LB}} (f;x) = - \int \ln \, \biggl( p(f) \, \sigma_f(x) \biggr) \, p[x] \, dx- \ln (\sqrt{2 \pi e}),$$ (2)

where p(f) is the probability, given the set of all stimuli, of a firing rate f, and $\sigma^2_f(x)$ is the variance of the firing rate in response to a given stimulus x.

To maximize the information transfer, does a neuron need to ``know" the arrival rates of photons impinging on the retina or the frequencies of sound waves hitting the ear's tympanic membrane? Since the ion channels in the dendrites only sense a voltage and not the stimulus directly, the answer to this question, fortunately, is no: maximizing the mutual information between the firing rate f and the dendritic voltage $V_{\text{dend}}(t)$ is equivalent to maximizing the information about the stimuli, as long as we can guarantee that the transformation from stimuli to firing rates is always one-to-one (9).

Since a neuron must be able to adapt to a changing environment and shifting intra- and extracellular conditions (4), learning and relearning of the proper conductance parameters, such as the channel densities, should occur on a continual basis. An alphabet zoo of different calcium ($\text{Ca}^{2+}$) conductances in neurons of the central nervous system, denoted `L', `N', `P', `R', and `T'-conductances, reflects a wealth of different voltage and pharmacological properties (10), matching an equal diversity of potassium ($\text{K}^+$) channels. No fewer than ten different genes code for various $\text{Ca}^{2+}$ subunits, allowing for a combinatorial number of functionally different channels (11). A self-regulating neuron should be able to express different ionic channels and insert them into the membrane. In information maximization, the parameters for each of the conductances, such as the number of channels, are continually modified in the direction that most increases the mutual information $I[f; V_{\text{dend}}(t)]$ each time a stimulus occurs (12).

The voltage and the conductances are nonlinearly coupled: the conductances affect the voltage, which, in turn, sets the conductances. Since the mutual information is a global property of the stimulus set, the learning rule for any one conductance would depend on the values of all other conductances, were it not for the nonlinear feedback loop between voltages and conductances. This nonlinear coupling must satisfy the strict physical constraint of charge conservation: when the neuron is firing periodically, the average current injected by the synaptic and voltage-dependent conductances must equal the average current discharged by the neuron (13). Remarkably, charge conservation results in a learning mechanism that is strictly local, so that the mechanism for changing one conductance does not depend on the values of any other conductances. For instance, information maximization predicts that the peak calcium or potassium conductance g_i changes by 

$$\Delta g_i = \eta(t) \Bigl\langle \frac{\delta}{\delta V_{\... ...nd}} \rangle) \, m_i h_i (E_i - V_{\text{dend}}) \Bigr\rangle$$ (3)

each time a stimulus is presented. Here $\eta(t)$ is a time-dependent learning rate, the angular brackets indicate an average over the stimulus duration, and $c(\langle V_{\text{dend}} \rangle)$ is a simple function that is zero for most commonly encountered voltages, equal to a positive constant below some minimum, and equal to a negative constant above some maximum voltage (14). This function represents the constraint on the maximum and minimum firing rate, which sets the limit on the neuron's dynamic range.

Given a stimulus x, the dominant term $\left\langle \displaystyle{\frac{\delta}{\delta V_{\text{dend}}(t)}} m_i h_i E_i\right\rangle$ of eq. 3 changes those conductances that increase the slope of the firing rate response to x. A higher slope means that more of the neuron's limited range of firing rates is devoted to representing the stimulus x and its immediate neighborhood. Since the learning rule is democratic yet competitive, only the most frequent inputs ``win" and thereby gain the largest representation in the output firing rate.

In Fig. 2a, the learning rule of eq. 3--generalized to also change the midpoint voltage and steepness of the activation and inactivation functions--has been used to train the model neuron as it responds to random, 200 msec long amplitude modulations of a synaptic input conductance to the dendritic compartment. The cell ``learns" the statistical structure of the input, matching its adapted firing rate to the cumulative distribution function of the conductance inputs. The distribution of firing rates shifts from a peaked distribution to a much flatter one, so that all firing rates are used nearly equally often (Fig. 2b). The information in the firing rate increases by a factor of three to 10.7 bits/sec, as estimated by adding a 5 msec, Gaussian-distributed noise jitter to the spike times.
 

  Figure 2:  

The inputs to the model are synaptic conductances,  drawn randomly from a Gaussian distribution of mean $\mu = 141 \; \text{nS}$and standard deviation of $\sigma = 25 \; \text{nS}$with the restriction that the conductance be non-negative (dot-dashed line).  The learning rule in eq. 3--maximizing the information in the cell's firing rate--was used  to adjust the peak conductances, midpoint voltages, and slopes of the ``dendritic" $\text{Ca}^{2+}$ and $\text{K}^+$ conductances over the course of $21.8 \; \text{(simulated) minutes}$. The learning rate decayed with time: $\eta(t) = \eta_0 \exp ({-t/\tau_{\text{learning}}})$,with $\eta_0 =$0.8 and $\tau_{\text{learning}} =$ 131 sec.  The optimal firing rate response curve (dotted line) is asymptotically proportional to the cumulative probability distribution of inputs. The inset illustrates the typical timecourse of the dendritic voltage in the trained model. The lower panel shows the probability distribution of firing rates before and after learning. Learning shifts the distribution from a peaked distribution to a much flatter one, so that the neuron uses each firing rate within the range [ 22 , 59 ] Hz equally often in response to randomly selected synaptic inputs.

Changing how tightly the stimulus amplitudes are clustered around the mean will increase or decrease the slope of the firing rate response to input, without necessarily changing the average firing rate. This gives rise to a specific experimental prediction that can be evaluated using standard in vitro current clamp methods: in Fig. 3, after the neuron has adjusted to one particular Gaussian probability distribution of stimuli, the same distribution is made narrower. The neuron responds to this change in its input distribution by increasing its gain.
 

  Figure 3:  

An experimental prediction. The model neuron is given a random current injection step every 200 msec (top row) over the course of an hour long experiment, to which the neuron learns to respond as shown in the center row. The amplitude of the current step is chosen from a Gaussian distribution with a mean at 0.45 nA and a standard deviation of 0.2 nA (bottom left). A sudden six-fold decrease in the standard deviation of the applied current finds the model neuron mismatched to the new ``stimulus environment." The adaptation rule of eq. 3, however, enables the neuron to adjust its dendritic conductances within 11 min, as can be seen by comparing the neuron's firing rate in response to the same stimulus current sequence before and after adaptation in the center row. The neuron's average response curve (bottom right) is considerably steeper now as a function of current, reflecting the increased output bandwidth accorded to the narrower input distribution.
 

While the detailed substrate for learning information maximization at both the single cell and network level awaits experimental elucidation, the terms in the learning rule of eq. 3 have simple biophysical correlates: the derivative term, for instance, is reflected in the stochastic flicker of ion channels switching between open and closed states. The transitions between simple open and closed states will occur at a rate proportional to ${\bigl[ {\displaystyle{d \over dV}} m_\infty(V) \bigr]}^\gamma$in equilibrium, where the exponent $\gamma$ is 1/2 or 1, depending on the kinetic model (15 ). To change the information transfer properties of the cell, a neuron could use state-dependent phosphorylation of ion channels or gene expression of particular ion channel subunits, possibly mediated by G-protein initiated second messenger cascades, to modify the properties of voltage-dependent conductances. The tools required to adaptively compress information from the senses are thus available at the subcellular level.

1/27/1998