year: 1962
paper: architectureofcomplexity-hsimon1962
website:
code:
connections: complexity, hierarchical, modular, coupling, cohesion, abstraction layers, representation, complexity theory
TLDR
Complex systems are almost universally hierarchic (subsystems within subsystems).
Hierarchic systems evolve far faster because stable intermediate forms preserve progress.
They also tend to be nearly decomposable: interactions within subsystems are much stronger than between subsystems.
The watchmaker parable
Two watchmakers build watches from 1000 parts. Tempus’s watches have no subassemblies: any interruption destroys all progress. Hora builds in modules of ~10, nested three levels deep. If interruptions arrive with probability per step, Hora is ~4000x faster.
This generalizes to any process where complex forms arise from simpler ones. Non-hierarchic complex forms don’t persist.
“Among possible complex forms, hierarchies are the ones that have the time to evolve.”
Near-decomposability
A system is nearly decomposable when interactions within subsystems are much stronger than interactions between subsystems.
→
(a) Short-run behavior of each subsystem is approximately independent of other subsystems.
(b) Long-run behavior depends on other subsystems only in an aggregate way.
Heat flow in a building
A perfectly insulated building divided into rooms (good insulation between rooms), each room subdivided into cubicles (poor insulation between cubicles). Start with random temperatures everywhere.
is the rate heat flows from cubicle to cubicle per degree of temperature difference between them. These heat-diffusion coefficients form a matrix:
A1 A2 A3 B1 B2 C1 C2 C3 A1 — 100 — 2 — — — — A2 100 — 100 1 1 — — — A3 — 100 — — 2 — — — B1 2 1 — — 100 2 1 — B2 — 1 2 100 — — 1 2 C1 — — — 2 — — 100 — C2 — — — 1 — 100 — 100 C3 — — — — 2 — 100 — Large values (100) cluster in blocks along the diagonal: within-room pairs. Cross-room values are small (1–2) or zero. If the off-diagonal blocks were exactly zero, this would be a block diagonal matrix: fully decomposable, rooms as independent systems. The small but nonzero cross-room values make it nearly decomposable.
Within hours, cubicles in each room equilibrate: the large within rooms mean heat flows fast across cubicle partitions. Rooms stay at different temperatures from each other. Over days, the small cross-room let heat leak between rooms until the whole building reaches a uniform temperature.
After the short-run equilibria settle, one thermometer per room suffices to describe the whole system.
Frequency separation
High-frequency dynamics within subsystems, low-frequency dynamics between them.
Molecular vibrations are faster than the interactions between molecules.
Team-internal communication is more frequent than cross-team coordination.
Higher-ups have longer planning horizons and less frequent interactions.
Once you condition on the aggregate variables, fine-grained microstates of different subsystems are approximately independent:
The aggregates form a sufficient statistic / markov blanket: they “screen off” fast intra-block detail from slow inter-block dynamics.
The empty world hypothesis
Most things are only weakly connected with most other things; for a tolerable description of reality only a tiny fraction of all possible interactions needs to be taken into account.
Understanding would require tracking every pairwise interaction.
If there are important systems in the world that are complex without being hierarchic, they may to a considerable extent escape our observation and our understanding.
Hierarchic systems are highly redundant
A few types of subsystems, many arrangements: proteins from 20 amino acids, molecules from ~90 elements), which is why they admit simple descriptions. A hierarchic system’s kolmogorov complexity can be far smaller than it looks.
How complex or simple a structure is depends critically upon the way in which we describe it”
Curricula roughly recapitulate the history of their field.
“ontogeny recapitulates phylogeny”
A DNA encodes not a blueprint of the final organism but a process that builds it stage-by-stage, each stage modifying the previous one. Process descriptions (recipes, differential equations) are often far simpler than state descriptions (blueprints, snapshots) of the same system.