From computer to brain : foundations of computational neuroscience.

Introduction 1 1.1 For whom is this book? . . . . . . . . . . . . . . . . . . . 1 1.2 What is in the book? . . . . . . . . . . . . . . . . . . . . 2 1.3 Do I need a computer for this book? . . . . . . . . . . . 3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Why learn this now? . . . . . . . . . . . . . . . . . . . . 4 1.5 What is the subtext? . . . . . . . . . . . . . . . . . . . . 5 1.6 How is the book organized? . . . . . . . . . . . . . . . . 6 I Perspectives 9 2 Computational Neuroscience and You 11 2.1 Why learn this? . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 Brainmetaphors . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 Compare and contrast computer and brain . . . . . . . . 12 2.4 Origins of computer science and neuroscience . . . . . . 14 2.5 Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Levels of organization . . . . . . . . . . . . . . . . . . . . 16 Levels of investigation . . . . . . . . . . . . . . . . . . . . 18 2.6 New engineering vs. old engineering . . . . . . . . . . . . 18 2.7 The neural code . . . . . . . . . . . . . . . . . . . . . . . 20 2.8 The goals and methods of computational neuroscience . 22 2.9 Summary and thoughts . . . . . . . . . . . . . . . . . . . 23 3 Basic Neuroscience 25 3.1 Why learn this? . . . . . . . . . . . . . . . . . . . . . . . 25 3.2 Microscopic view of the nervous system . . . . . . . . . . 26 3.3 Macroscopic view of the nervous system . . . . . . . . . 29 Slicing the brain . . . . . . . . . . . . . . . . . . . . . . . 29 3.4 Parts of the brain . . . . . . . . . . . . . . . . . . . . . . 31 3.5 How do we learn about the brain? . . . . . . . . . . . . . 34 Anatomical methods . . . . . . . . . . . . . . . . . . . . 35 3.6 Neurophysiology . . . . . . . . . . . . . . . . . . . . . . . 36 3.7 Molecular biology and neuropharmacology . . . . . . . . 37 3.8 Psychophysics . . . . . . . . . . . . . . . . . . . . . . . . 38 3.9 Clinical neurology and neuropsychology . . . . . . . . . . 39 Ablative diseases . . . . . . . . . . . . . . . . . . . . . . 39 Intrinsic diseases . . . . . . . . . . . . . . . . . . . . . . . 40 3.10 Summary and thoughts . . . . . . . . . . . . . . . . . . . 41 II Computers 43 4 Computer Representations 45 4.1 Why learn this? . . . . . . . . . . . . . . . . . . . . . . . 45 4.2 Calculator or typewriter . . . . . . . . . . . . . . . . . . 47 4.3 Punch cards and Boolean algebra . . . . . . . . . . . . . 48 4.4 Analog vs. digital representations . . . . . . . . . . . . . 50 4.5 Types of computer representations . . . . . . . . . . . . 51 4.6 Representation of numbers . . . . . . . . . . . . . . . . . 52 Representation of letters and words . . . . . . . . . . . . 55 4.7 Representation of pictures . . . . . . . . . . . . . . . . . 56 4.8 Neurospeculation . . . . . . . . . . . . . . . . . . . . . . 59 4.9 Summary and thoughts . . . . . . . . . . . . . . . . . . . 62 5 The Soul of an Old Machine 63 5.1 Why learn this? . . . . . . . . . . . . . . . . . . . . . . . 63 5.2 The art of the hack . . . . . . . . . . . . . . . . . . . . . 64 5.3 Software and hardware . . . . . . . . . . . . . . . . . . . 65 5.4 Basic computer design . . . . . . . . . . . . . . . . . . . 66 Pointers come fromcomputermemory design . . . . . . 67 Sequential algorithms come from computer control flow . 67 CPU:machine commands . . . . . . . . . . . . . . . . . 69 5.5 Programs and hacks . . . . . . . . . . . . . . . . . . . . . 69 Conditionals . . . . . . . . . . . . . . . . . . . . . . . . . 71 5.6 Pointermanipulation . . . . . . . . . . . . . . . . . . . . 74 A kludge . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 A computer virus . . . . . . . . . . . . . . . . . . . . . . 76 5.7 Neurospeculation . . . . . . . . . . . . . . . . . . . . . . 77 5.8 Summary and thoughts . . . . . . . . . . . . . . . . . . . 82 III Cybernetics 83 6 Concept Neurons 87 6.1 Why learn this? . . . . . . . . . . . . . . . . . . . . . . . 87 6.2 History and description of McCulloch-Pitts neurons . . . 88 6.3 Describing networks by weights and states . . . . . . . . 90 Calculating total-summed-input by dot product . . . . . 92 Calculating state . . . . . . . . . . . . . . . . . . . . . . 94 6.4 From single unit to network of units . . . . . . . . . . . . 95 6.5 Network architecture . . . . . . . . . . . . . . . . . . . . 102 6.6 Summary and thoughts . . . . . . . . . . . . . . . . . . . 104 7 Neural Coding 105 7.1 Why learn this? . . . . . . . . . . . . . . . . . . . . . . . 105 7.2 Coding in space: ensemble codes . . . . . . . . . . . . . . 106 Local vs. distributed ensemble coding . . . . . . . . . . . 108 7.3 Coding with volts and chemicals: neural state code . . . 110 7.4 Coding in time: temporal and rate codes . . . . . . . . . 111 Temporal integration . . . . . . . . . . . . . . . . . . . . 112 Clocking . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 7.5 Frequency coding . . . . . . . . . . . . . . . . . . . . . . 114 7.6 Summary and thoughts . . . . . . . . . . . . . . . . . . . 118 8 Our Friend the Limulus 121 8.1 Why learn this? . . . . . . . . . . . . . . . . . . . . . . . 121 8.2 The biology . . . . . . . . . . . . . . . . . . . . . . . . . 122 8.3 What we can ignore . . . . . . . . . . . . . . . . . . . . . 122 8.4 Why the eye lies: the problem . . . . . . . . . . . . . . . 123 8.5 Design issues . . . . . . . . . . . . . . . . . . . . . . . . . 126 Making themodel small—scaling . . . . . . . . . . . . 126 Making the model small — dimensional reduction . . . . 127 Eliminating edge effects—wraparound . . . . . . . . . . 128 Presenting the input — parameterization . . . . . . . . . 130 Parameterizing the activation function . . . . . . . . . . 132 Parameterizing the weight matrix . . . . . . . . . . . . . 132 8.6 The limulus equation . . . . . . . . . . . . . . . . . . . . 134 8.7 State calculation . . . . . . . . . . . . . . . . . . . . . . . 135 8.8 Life as a limulus . . . . . . . . . . . . . . . . . . . . . . . 137 8.9 Summary and thoughts . . . . . . . . . . . . . . . . . . . 139 9 Supervised Learning: Delta Rule and Back-Propagation 141 9.1 Why learn this? . . . . . . . . . . . . . . . . . . . . . . . 141 9.2 Supervised learning . . . . . . . . . . . . . . . . . . . . . 142 9.3 The delta rule . . . . . . . . . . . . . . . . . . . . . . . . 145 The energy analogy . . . . . . . . . . . . . . . . . . . . . 146 The delta rule solves AND . . . . . . . . . . . . . . . . . 147 9.4 Backward propagation . . . . . . . . . . . . . . . . . . . 149 9.5 Distributed representations . . . . . . . . . . . . . . . . . 151 9.6 Distributed representation in eye movement control . . . 152 Design of themodel . . . . . . . . . . . . . . . . . . . . . 154 Results fromthemodel: generalization . . . . . . . . . . 157 Exploration of themodel: hidden unit analysis . . . . . . 159 Computer modeling vs. traditional mathematical modeling 161 9.7 Summary and thoughts . . . . . . . . . . . . . . . . . . . 162 10 Associative Memory Networks 163 10.1 Why learn this? . . . . . . . . . . . . . . . . . . . . . . . 163 10.2 Memories in an outer product . . . . . . . . . . . . . . . 164 Association across a single synapse . . . . . . . . . . . . 164 The outer product of two vectors . . . . . . . . . . . . . 165 Making hetero- and autoassociativememories . . . . . . 167 Limit cycles . . . . . . . . . . . . . . . . . . . . . . . . . 171 Instantaneous vs. gradual learning and recall . . . . . . . 174 10.3 Critique of the Hopfield network . . . . . . . . . . . . . . 176 10.4 Summary and thoughts . . . . . . . . . . . . . . . . . . . 177 IV Brains 179 11 From Soap to Volts 189 11.1 Why learn this? . . . . . . . . . .