Most recent languages present a forcible transfer of control out of the block in which the exception occurred (Ada, Java), and some languages just give you considerable rope to hang yourself (C/UNIX setjmp longjmp). [PL/I and perhaps Basic try to present more options?]
One major problem is that compilers, especially optimizing compilers may have substantially altered the sequence of operations so that you can't go back to any recognizable place, and even if you could, you may not have any reasonable linguistic access to pertinent information (either in registers, implicit in the program counter, etc.) Unless the programmer explicitly backed up the computation parameters in globally accessible memory.
Implicit control-flow changes in conventionally compiled languages present difficulty with both language scope and control. (Unusual in this regard are those few languages like Scheme with first-class data types of Continuations, where the notion ``continuation from this point on'' can be represented as data and manipulated.)
There is also the considerable difficulty faced by the programmer in writing secure code that will guard against untoward exits to unintended exception handlers (cf. Ariane rocket disaster). Even the assumption of default handlers must be reinforced (e.g. Java signatures).