You didn't specify the processor or the compiler you are using.
On any of the x86 family and and compiler I'm aware of, the answer would be that overflow has no impact on performance whatsoever. This is probably true for other processors as well.
If you want to be sure, look at the assembly code that is produced (-S in clang or gcc, make sure there is a -g or -ggdb3 so it includes the source code line numbers), and you will probably find it translates to something like (produced with gcc on x86-64):
subl %esi, %edi
andl %edx, %edi
If you do the same thing with signed,
subl %esi, %edi
andl %edx, %edi
So there is no difference, and if you want to be complete about it, look up the instruction timing for your processor (https://www.agner.org/optimize/instruction_tables.pdf is great for this for x86) to check that there is no timing penalty for setting the overflow carry flag (I've never seen a processor for which this is the case). Under the hood, the difference between signed and unsigned usually only matters when doing something like inequality, multiplication, division or converting to and from strings.
Adding in a specific test will either make it slower, or the compiler will optimize the test out if it sees that both sides do the same thing.
Having said that, you risk running into undefined behavior when you try this sort of thing (even if this particular case is not undefined) and should at the very least add a comment about what you are doing so this doesn't trip you up later.