It compromises speed in several ways, especially if compatibility requirements force machines that are inherently faster (and more precise), and have more memory and registers, to be run slowly so as to match the results of less-precise calculations.
It forces control or calculation regimes that are suitable for slow computers on ones that are able to run faster. That is, one could in principle run all numeric programs entirely portably on each succeeding generation of supercomputer by disabling all new features and emulating the previous generation, bit for bit. Eliminating calculation restructuring that might change expressions in small ways for big cache improvements.
We don't want to do that. What might be a more reasonable approach is to strive for as much accuracy as possible in each circumstance (within the constraints of time and space), and hope that two different platforms each striving toward the same goal, will be substantially the same.