Reducing Inherent Communication
Communication is expensive!
Measure: communication to computation ratio
Inherent communication
- Determined by assignment of tasks to processes
- One produces data consumed by others
=> Use algorithms that communicate less
=> Assign tasks that access same data to same process
- same row or block to same process in each iteration