Good news! A colleague had a different issue and gave me this as a solution:
It was completely not what my error was, but I followed the instructions to downgrade wsl from the latest (2.5.7) to 2.4.13.
- Removed all distributions with wsl --unregister
- Uninstalled wsl
- Rebooted
- Uninstalled Docker Desktop
- Rebooted
- Installed wsl 2.4.13 : Release 2.4.13 · microsoft/WSL · GitHub
- Installed Docker Desktop
Unfortunately after that I still had a problem starting Docker Desktop. I had to remove all distributions again. Then started Docker Desktop again.