Types of Contention
- Module: all-to-all personalized comm. in matrix transpose
- solution: stagger access by different processors to same node temporally
- In general, reduce burstiness; may conflict with making messages larger
Network contention and end-point contention (hot-spots)
Location and Module Hot-spots
- Location: e.g. accumulating into global variable, barrier
- solution: tree-structured communication