High latency of sending data between containers that are running on LAN

I am running 2 containers that the first one will send data to the second one by using TCP programming. Then I tried to measure the latency of this process ( 1st container also sends its starting time to the 2nd container, which will compare to its current time to get the latency result). I noticed that the latency value when i ran 2 containers on the same host is always hundreds millisecond more than the case when i ran each container on different hosts on a local network ( via switch).
I’d used NTP to synchronize both hosts, so the difference in time may not exceed a hundred millisecond in my opinion. Some websites i’ve read state that the latency between two hosts in a LAN stays around several milliseconds, is it true?
Can anyone lists here some other factors which cause this high latency value?
Thank you for reading.

p/s: the packet that i send is a 1 MB JPEG image, 6 images per second, so it cannot be throttled by my switch bandwidth.