It was more of a hunch, than actually knowing what causes the problem. I always wondered whether it makes a difference whether in-box (what is installed by the feature) or the application (either from the Microsoft store or GitHub release page) makes a difference. Apparently it does not.
You might want to check this topic, as it has a couple of suggestions how to fix it: Unable to install Docker Desktop 4.8.2: Component CommunityInstaller.EnableFeaturesAction failed: Not found