is it possible to design a cache for a CPU that doesn't add latency to cache misses (compared to the latency if there was no cache)?

That would be a cache with simultaneous lookup (send the request to multiple places at the same time). It's an effective strategy for looking up multiple levels of the cache, e.g. L2, L3, but it would likely overload the memory controller and make average performance worse.

thanks