Windows Docker container intranet access fails after 6 minutes of the server reboot

General Discussions
1

Expected behavior

The container should fetch data from the internal network SQL server and pass it to cloud.

Actual behavior

The container works normal after the server is rebooted. Exactly after 7 minutes it is getting no network path found with the internal network server and no longer getting data from the SQL server to pass it to cloud.

Information

[17:17:48 INF] [dc961cfc-627c-4e3d-bb27-f3e451d41126|c73eda92-3f1e-4ed7-bf53-7fa3c57e7eee] Collecting real time agent states
[17:18:10 ERR] [dc961cfc-627c-4e3d-bb27-f3e451d41126|c73eda92-3f1e-4ed7-bf53-7fa3c57e7eee] CI.Connectors.RealTime.Import.Cisco.UCCEV9.Application.Services.StateFetchingService failed.
CI.Connectors.Common.ConnectorException: CI.Connectors.RealTime.Import.Cisco.UCCEV9.Application.Services.StateFetchingService failed.
 ---> Microsoft.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)
 ---> System.ComponentModel.Win32Exception (53): The network path was not found.
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnectionString connectionOptions, Boolean withFailover)
   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover)
   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)
   at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry, SqlConnectionOverrides overrides)
   at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)
   at CI.Connectors.RealTime.Import.Cisco.UCCEV9.CiscoRtaDataCollector.GetStateRecords() in C:\src\CI.Connectors.RealTime.Import.Cisco.UCCEV9\CiscoRtaDataCollector.cs:line 49
   at CI.Connectors.RealTime.Import.Cisco.UCCEV9.Application.Services.StateFetchingService.FetchStateRecords() in C:\src\CI.Connectors.RealTime.Import.Cisco.UCCEV9\Application\Services\StateFetchingService.cs:line 68
ClientConnectionId:00000000-0000-0000-0000-000000000000
Error Number:53,State:0,Class:20
   --- End of inner exception stack trace ---

Windows server 2019 standard
Docker version:

PS C:\Windows\system32> docker version
Client: Mirantis Container Runtime
 Version:           20.10.9
 API version:       1.41
 Go version:        go1.16.12m2
 Git commit:        591094d
 Built:             12/21/2021 21:34:30
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Mirantis Container Runtime
 Engine:
  Version:          20.10.9
  API version:      1.41 (minimum version 1.24)
  Go version:       go1.16.12m2
  Git commit:       9b96ce992b
  Built:            12/21/2021 21:33:06
  OS/Arch:          windows/amd64
  Experimental:     false

Steps to reproduce the behavior

  1. … Restart the server and wait for 7 minutes
  2. …
2

This is not an usual behavior of Windows containers. I’m sure there’s more to the environment that is causing it. Can you please elaborate on where this is running? Is it a single Windows Server? Are you using K8s or something else? Any additional info might help investigate this.

3

It is running on a Windows server 2019 standard VM server.

4

May i know where i can find debug logs?

5

You should be able to see the logs on Event Viewer. Alternatively, you can use the Log Monitor tool to pass all logs and event to STDOUT: windows-container-tools/LogMonitor at main · microsoft/windows-container-tools (github.com)

To clarify: Are you saying that the app works for a few minutes and then the error above happens?

6

Thank you! yes, The container collect logs from the database server which is in the local network and send it to cloud for first 7 minutes when the server rebooted. then error are coming.

7

Got it. Sorry, one more clarification: The error happens because of the reboot, or does it happen before the reboot and by rebooting it you can solve the problem?

Have you looked into Event Viewer to see if there’s anything there? You’ll need Log Monitor to see the logs from inside the container. But first check the logs from the host.

8

Alternatively, I wrote a blog post to help look at the event viewer inside the container: How to manually troubleshoot applications on Windows containers - Microsoft Community Hub

9

no worries. thank you for your time to take a look of my messages. It only works for just 7 minutes after reboot then throughing error and no longer working.
Let me look at your blog

10

This sounds similar to an issue I experienced with Oracle. It seemed to be a Microsoft issue. We ended up dropping back to Server 2016 for now.

1 Like