video_core/memory_manager: Avoid repeated unnecessary page slot lookups

We don't need to keep calling the same function over and over again in a
loop, especially when the behavior is slightly non-trivial. We can just
keep a reference to the looked up location and do all the checking and
assignments based off it instead.