core/loaders: Simplify PhysicalMemory usage.

It is currently a std::vector, however we might want to replace it with a more fancy allocator.
So we can't use the C++ iterators any more.