Is it possible to avoid the load metadata step on a build?

I’m working on some optimizations, and finding it frustrating to see almost a full second on every build attempting to load metadata for an image I already have downloaded. Is there any way to prevent this?