ANN Foundations
Source: Unit 2 §4
An artificial neural network is a computing model borrowed wholesale from the brain. Before any of the maths, it is worth seeing the thing being copied.
The biological inspiration
| Biological part | Role |
|---|---|
| Dendrite | Receives signals from other neurons |
| Synapse | Point of connection to other neurons |
| Soma | Processes the information |
| Axon | Transmits the output of this neuron |
- The brain is a massively parallel interconnected network of neurons, with the sense organs relaying information into the lowest layer.
- An average human brain has roughly neurons - 100 billion.
- Parallelism buys division of work: each neuron plays one role, responding to one certain stimulus.
What ANNs are good for
Neural networks provide a robust approach to approximating real-valued, discrete-valued and vector-valued functions.
- Handwritten character recognition.
- Spoken-word recognition.
- Face recognition.
- Self-driving cars.
- Generally: interpreting real-world sensor data, which is noisy and high-dimensional and where no clean rule exists to write down.
When to use an ANN
Instances are described by many attribute-value pairs→ANN fitsit takes a wide input vector natively
The target is discrete-valued, real-valued, or a vector of such values→ANN fitsthe output layer can be any shape
The training examples may contain errors→ANN fitsnetworks are robust to noise
Long training times are acceptable→ANN fitstraining is the expensive half; prediction is cheap
A human must be able to read and justify the learned rule→Pick a decision tree insteadANN weights are not interpretable
An ANN is the right answer when the question is "learn a function from a lot of noisy numeric evidence" and the wrong answer when the question is "explain the rule you learned".