renderer_{opengl,vulkan}: Clamp shared memory to host's limit

This stops shaders from failing to build when the exceed host's shared
memory size limit. An error is logged.