Have you enabled NUMA spanning in your HyperV settings? if not enable it i bet that will solve your issue.
By default, Windows Server enables NUMA spanning, which provides the most flexibility as virtual machines (VMs) can access and use memory in any NUMA node. But it may result in lower performance compared to forcing VMs to use memory on the same NUMA node as the processor cores.
By disabling NUMA spanning, you ensure that VMs use memory and processor cores in the same NUMA node, giving the best performance.
This should only be changed once, if, as an administrator, you feel comfortable with NUMA and the implications of disabling and also if you have some additional management suite that can help ensure best configuration.
To configure NUMA spanning, open the Hyper-V Settings and select the NUMA Spanning option and disable it, I am sure, it will get solved; I struggled with the issue for a week and resolved it by disabling NUMA.
I am sure this would be marked as resolved by disabling NUMA in Hyper-V Manager.