Extended Hierarchy
Idealized view: local cache hierarchy + single main memory
But reality is more complex
- Centralized Memory: caches of other processors
- Distributed Memory: some local, some remote; + network topology
- Management of levels
- caches managed by hardware
- main memory depends on programming model
- SAS: data movement between local and remote transparent
- message passing: explicit
- Levels closer to processor are lower latency and higher bandwidth
- Improve performance through architecture or program locality
- Tradeoff with parallelism; need good node performance and parallelism