buffer_cache: Only flush downloaded size

Fixes a regression unintentionally introduced by the garbage collector.
This makes regular memory downloads only flush the requested sizes.

This negatively affected Koei Tecmo games.