direct quotes:
- all scientific endeavor is based, to a greater or lesser degree, on the existence of universality,
- the study of complex systems as a new endeavor strives to increase our ability to understand the universality that arises when systems are highly complex.
- Sometimes universal principles are intuitively appreciated without being explicitly stated.
- how the complexity of the whole is related to the complexity of the parts.
- Can we describe a system composed of simple parts where the collective behavior is complex? This is an important possibility, called emergent complexity.
- Can we describe a system composed of complex parts where the collective behavior is simple? This is also possible, and it is called emergent simplicity.
- (according to information & computation theory,) complexity is the amount of information necessary to describe a system.
- Our purpose in studying complex systems is to extract general principles.
- avoid the “cute picture” syndrome, where pictures are presented without accompanying discussion or analysis. (cute picture syndrome).
- A complex system is a system formed out of many components whose behavior is emergent, that is, the behavior of the system cannot be simply inferred from the behavior of its components. The amount of information necessary to describe the behavior of such a system is a measure of its complexity.
- global emergence-where collective behavior pertains to the system as a whole.
- a gas of particles. Two emergent properties of a gas are its pressure and temperature. The reason they are emergent is that they do not naturally arise out of the description of an individual particle. We generally describe a particle by specifying its position and velocity. Pressure and temperature become relevant only when we have many particles together.
- Assume for the moment that the pattern of firing represents a sentence, such as “To be or not to be, that is the question:’ We can recover the complete sentence by pre- senting only part of it to the network “To be or not to be, that” might be enough. We could use any part to retrieve the whole, such as, “to be, that is the question:’ This kind of memory is to be contrasted with a computer memory, which works by assigning an address to each storage location. To access the information stored in a particular location we need to know the address. In the neural network memory, we specify part of what is located there, rather than the analogous address: Hamlet, by William Shakespeare, act 3, scene 1, line 64. More central to our discussion, however, is that in a computer memory a particular bit of information is stored in a particular switch. By contrast, the network does not have its memory in a neuron. Instead the memory is in the synapses. In the model, there are synapses between each neuron and every other neuron. If we remove a small part of the network and look at its properties, then the number of synapses that a neuron is left with in this small part is only a small fraction of the number of synapses it started with. If there are more than a few patterns stored, then when we cut out the small part of the network it loses the ability to remember any of the patterns, even the part which would be represented by the neurons contained in this part.