You may be facing the same problem as me, and your Docker Desktop is using 70%+ CPU on macOS. And that’s without even running a container.
I searched for a solution for a long time, but in the end, it was quite simple:
Switch the Virtual Machine Manager (VMM) to HyperKit.
For me, it was set to the Apple Virtualization Framework by default, which obviously doesn’t work well.
Note:
I’m using a 2019 Intel MacBook Pro