swap: Remove unused methods

Also gets rid of pointer data variants as this prevents the use of
the regular swapping routines as unary predicates in std lib functions.

They also cast to stricter alignment types, which is undefined behavior.