The theory here feels a lot similar to quantum computing, where you "only" pay "significant" costs when you want to interact with / latch signals for the world outside your circuit

Well, in quantum computing, even the simplest gates require an enormous overhead to implement at present.