Analysis Framework
Solving communication and load balance NP-hard in general case
- But simple heuristic solutions work well in practice
Fundamental Tension among:
- balanced load
- minimal synchronization
- minimal communication
- minimal extra work
Good machine design mitigates the trade-offs