Yes, I did a radical manual clean of the registry and all files I could find (%localappdata% and C:\Program Files\Docker), rebooted. Also, I vaguely remember that I discovered the problem was apparently caused by the update installer being located in the same folder as the files the updater tries to delete. So it is vital that the installer is run from a different folder.
All is running smoothly now. Thanks for checking back with me.