Container connection get disturbed in docker

Hi Team

We are getting the connection disturbance inside the docker container and container’s are getting down. once the container getting down its getting up only by restarting the services.
We tried to find out the exit code some how we are unable to get the container id of stop containers as server was rebooted. Please find below docker info :

#docker info
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
scan: Docker Scan (Docker Inc., v0.7.0)

Server:
Containers: 10
Running: 4
Paused: 0
Stopped: 6
Images: 83
Server Version: 20.10.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: false
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: yhfkdls7dbyati0f89zd409oq
Is Manager: true
ClusterID: yx1fv317vkgav0elddnydfac2
Managers: 6
Nodes: 9
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Data Path Port: 4789
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 10.240.129.68
Manager Addresses:
10.240.129.68:2377
10.240.129.70:2377
10.240.129.72:2377
10.240.129.72:2377
10.240.129.73:2377
10.240.129.74:2377
Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.4.0-1051-azure
Operating System: Ubuntu 18.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.64GiB
Name: java-pr-server1
ID: L53L:AVHX:A3NZ:TSSO:25VC:43U6:LMQ5:CHG5:W23N:HSFB:5UTL:NLQH
Docker Root Dir: /docker_data
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support
root@java-pr-server1:~#

If any further information needed then let me know.

Regards
Anugrah Verma

We’ll definitely need more information. Can you specify what you mean by “connection disturbance”? Like network connections? Connections to what?

And what “services” exactly are you restarting to restore functionality?

Is the Docker systemd service stopping? Is there anything in the Docker logs (journalctl -u docker)?

Internet is not working suddenly in Docker container, we need to restart the container, then only everything seems working fine

{“instant”:{“epochSecond”:1626688110,“nanoOfSecond”:182000000},“thread”:“http-nio-80-exec-7”,“level”:“ERROR”,“loggerName”:“org.apache.c
atalina.core.ContainerBase.[Tomcat].[localhost].[/].[dispatcherServlet]”,“message”:“Servlet.service() for servlet [dispatcherServlet] i
n context with path threw exception [Request processing failed; nested exception is java.lang.RuntimeException: java.lang.Exception:
org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 : [{“timestamp”:“2021-07-19T09:48:30.154+00:00”,
“status”:500,“error”:“Internal Server Error”,“message”:”",“path”:"/prefs"}]] with root cause",“thrown”:{“commonElementCoun
t”:0,“localizedMessage”:“500 : [{“timestamp”:“2021-07-19T09:48:30.154+00:00”,“status”:500,“error”:“Internal Server Error”,“m
essage”:”",“path”:"/prefs"}]",“message”:“500 : [{“timestamp”:“2021-07-19T09:48:30.154+00:00”,“status”:500,“error”:“Inte
rnal Server Error”,“message”:”",“path”:"/prefs"}]",“name”:“org.springframework.web.client.HttpServerErrorException$InternalSer
verError”,“extendedStackTrace”:“org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 : [{“timestamp”:“20
21-07-19T09:48:30.154+00:00”,“status”:500,“error”:“Internal Server Error”,“message”:”",“path”:"/prefs"}]\n\tat org.sprin
gframework.web.client.HttpServerErrorException.create(HttpServerErrorException.java:100) ~[spring-web-5.2.6.RELEASE.jar!/:5.2.6.RELEASE
]\n\tat org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:172) ~[spring-web-5.2.6
.RELEASE.jar!/:5.2.6.RELEASE]\n\tat org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.
java:112) ~[spring-web-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]\n\tat org.springframework.web.client.ResponseErrorHandler.handleError(Respons
eErrorHandler.java:63) ~[spring-web-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]\n\tat org.springframework.web.client.RestTemplate.handleResponse
(RestTemplate.java:782) ~[spring-web-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]\n\tat org.springframework.web.client.RestTemplate.doExecute(Res
tTemplate.java:740) ~[spring-web-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]\n\tat org.springframework.web.client.RestTemplate.execute(RestTempl
ate.java:674) ~[spring-web-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]\n\tat org.springframework.web.client.

I see an error of 500 returned or received. If this is a 500 HTTP Response code being returned to some client, then what is that API endpoint doing which would make it respond in that way? I don’t know the code, so you will need to follow the stack trace yourself.

I’m not convinced it’s a network issue as the logs you showed only prove that the HTTP server you have running responded with a 500 HTTP response code. If you really think it’s a network issue, you can follow up with a bit more debugging. Can you query DNS using dig or nslookup against 1.1.1.1 or 8.8.8.8 nameservers? Can you ping google.com?

If those all fail, maybe provide the configuration for the Docker container (docker inspect) along with some Docker daemon logs journalctl -u docker–try the pin point the time when this happens, please)