I found this comment:
RobG [Over a year ago]
If you run with Hyperv Isolation then the language is fixed as en-US with ANSI System Locale set to UTF-8 (65001). There is no known way to change this behaviour in hyperv
Another comment below that shares an issue in the Moby repository (The base of Docker CE)
brixenDK [Over a year ago]
Adding to what @RobG says: They are aware of the issue but it is a significant change to support it github.com/moby/moby/issues/40535#issuecomment-930650879
At least if I understood your problem, this could be related to that