Programmatic manipulation and management of docker-desktop

Hello guys, I’m here because I have a personal project in hand, which revolves around docker-desktop.

I am developing in C#/Winform for .net8.0 a user interface that allows for the automated deployment of a LAMP server system.

It is something similar to what configuration scripting would be but at the docker level.

The complexity that I am facing right now is that to correctly deploy and/or the characteristics that I need from the lamp server system, I need to activate/modify some settings in docker desktop (I don’t know if it can be done through direct commands to the Docker engine) to achieve The task I am working with is first:

C:\Users<USERNAME>\AppData\Roaming\Docker\setting.json

I programmatically alter the json making these changes:

Dictionary<string, string> settingsToReplace = new()
{
    { "\"exposeDockerAPIOnTCP2375\": false,", "\"exposeDockerAPIOnTCP2375\": true," },
    { "\"updateHostsFile\": false,", "\"updateHostsFile\": true," },
    { "\"displayedOnboarding\": false,", "\"displayedOnboarding\": true," },
    { "\"licenseTermsVersion\": 0,", "\"licenseTermsVersion\": 2," },
    { "\"noWindowsContainers\": true,", "\"noWindowsContainers\": false," },
    { "\"runWinServiceInWslMode\": false,", "\"runWinServiceInWslMode\": true," }
};

foreach (var setting in settingsToReplace)
{
    if (jsonContent.Contains(setting.Key))
    {
        jsonContent = jsonContent.Replace(setting.Key, setting.Value); // with this I exchange the json values
        update = true;
    }
}

So far everything is going well… the first real problem is restarting the docker-desktop environment and all its services or those that are necessary for it to recognize the new settings.

I have tried to run the following commands:

string dockerDesktopPath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ProgramFiles), @"Docker\Docker\Docker Desktop.exe");

await run.RunCommand("Stop-Process -Name \\\"Docker Desktop\\\"", "powershell",true);
await run.RunCommand("Stop-Service -Name \\\"com.docker.service\\\"", "powershell",true);
await run.RunCommand("Start-Service -Name \\\"com.docker.service\\\"", "powershell",true);
await run.RunCommand("Start-Process -FilePath \\\"" + dockerDesktopPath + "\\\"", "powershell",true);

from the C# application and tried to execute the following commands, calling them through a PowerShell instance:

public async Task RunCommand(string script, string command = "powershell", Boolean show = false, string? workDirectory = null)
{
string arguments = $"-Command \"Set-ExecutionPolicy -Scope Process -ExecutionPolicy Unrestricted -Force ; {script}";

ProcessStartInfo processStartInfo = new(command)
{
    Arguments = arguments,
    RedirectStandardOutput = true,
    RedirectStandardError = true,
    UseShellExecute = false,
    Verb = "runas",
    CreateNoWindow = true
};

Process process = new() { StartInfo = processStartInfo };

process.OutputDataReceived += (sender, e) =>
{
    if (!string.IsNullOrEmpty(e.Data))
    {
        //write on console for tracking
    }
};

process.ErrorDataReceived += (sender, e) =>
{
    if (!string.IsNullOrEmpty(e.Data))
    {
        //write on console for tracking
    }
};

process.Start();
process.BeginOutputReadLine();
process.BeginErrorReadLine();

await process.WaitForExitAsync();
}

The problem I am having is that when I execute the commands, and open docker-desktop, it becomes completely unstable, to the point that the docker-desktop graphical interface crashes.

1 Like

My Docker desktop runs stable with these settings:

  "exposeDockerAPIOnTCP2375": true,
  "updateHostsFile": false,
  "displayedOnboarding": true,
  "licenseTermsVersion": 2,
  "noWindowsContainers": false,
...
  "runWinServiceInWslMode": false,
  "wslEngineEnabled": true,
  "wslInstallMode": "installLatestWsl",
  "wslUpdateRequired": false

So I would have to guess and say that the option “runWinServiceInWslMode” for WSL version 1 is and should be “false”. And there seems to be an option “wslEngineEnabled” with the value “true” for WSL 2.

Or as Microsoft says:

We recommend that you use WSL 2 as it offers faster performance and 100% system call compatibility

Or does your Docker run stably with these settings when the Powershell script is not running?

The unstable thing is not related to the configuration… apparently it is related to the correct way to programmatically close and reopen the docker desktop, this is necessary for it to reload the new configuration. ok, I don’t know if there is any alternative to change the settings programmatically or from powershell and do a reload…

I have detected that despite enabling, for example, port 2375 or any other parameter in the setting.json, I must close the application and open it again, this is not functional, if I want to do a programmatic installation, I am not achieving it.

If you change the settings manually and then call

restart-service *docker*

in Powershell with administrator rights, this should work without any problems.

And if you do exactly the same with C#, then I see no reason why Docker Desktop should be unstable.
Maybe you can try only to restart the service in Powershell with administrator rights.

In your programming I see “runas” but it is not obvious to me if you stop and start the services with administrator rights. This could cause the problem.

Unfortunately my knowledge of C# is very rudimentary.

I am also not sure in which user context the process is restarted. If you manually open a console as a user with administrator rights, it is clear. But if you start a program that then executes a command in a shell, then it could also be that it happens in the system context. This could also cause the problem.

Why don’t you skip (comment out) the start of the process and instead stop it with a breakpoint at this point in the debugger and then run it manually with Powershell? Does it run stable then? And then do the same with the service. That would certainly help to narrow down the problem.

And if the way the Powershell commands are executed via C# is wrong, then you should ask about it in an appropriate forum.

I have just realized that you use asynchronous programming. But I don’t think that makes sense here. Each function or method must be completed before the next one can begin. This could very probably cause problems. So synchronous programming would make more sense.

Maybe this will help:
https://www.google.com/search?q=WaitForExitAsync+hangs

everything I’m doing I do with elevated privileges… and everything means everything that is C# and Powershell, tell me if it’s still not clear.

The problem is not the script… I am working with version 4.27.1 (136059) and apparently it has a very serious error and I don’t know if it is my speculation or little knowledge but when updating:

“updateHostsFile”: true,

I don’t understand why despite restarting the PC… this field is still not checked:

image

and I’m in WSL2

image

I checked my Hosts file and it’s empty…

Under which Windows user is the C# program running? And is this user a member of the “docker-users” group? There are two users on my computer, an unprivileged user who is a member of the “docker-users” group and a privileged administrator. And keep in mind:

The privileged helper com.docker.service is a Windows service which runs in the background with SYSTEM privileges. It listens on the named pipe //./pipe/dockerBackendV2 . The developer runs the Docker Desktop application, which connects to the named pipe and sends commands to the service. This named pipe is protected, and only users that are part of the docker-users group can have access to it.

Under this version there is a problem where the default settings values were being reset to default on startup, when using Settings Management. You should update to the new version.

Why not try it the other way around? Make a copy of the Docker configuration files or use git to monitor changes to these files. Make changes in the GUI. Restart everything. Check if it worked. And then look for the differences in the configuration files. Unfortunately, I don’t have time to do this myself at the moment. Sorry.

I would also check every change in the GUI first. Only then I would make every change with Powershell alone and see if everything works and at least I would make changes with C# + Powershell. This way it is also clear whether it is a Docker-related problem, a Powershell-related problem or a problem in C# programming.

1 Like

Hello, blue… I’m sorry to tell you that asking more questions about what I’m doing or not is not helping me. If you have more questions based on presumptions… can you make a complete summary of them to answer them???

I have followed the steps that have been indicated to me:

  1. install the application with my Windows 11 users, and when doing so do so with elevated privileges (as Administrator), run as administrator, if you are not familiar with this, it is the maximum level of possible permissions used by an administrator user at time of execution.

  2. open the docker desktop application and do the suvey.

  3. close the application manually (cheked on try icon).

  4. make a backup of setting.json.

  5. run the script to modify the setting.json file (i am runing this from terminal open/with elevated privileagies).

  6. Compare the bakcup file and the new one with the changes.

  7. Open the application and compare the new setting file with the data marked in the settings menu.

I already read the documentation that you gave me approximately 15 days ago and it has not helped me at all… a lot of text and few images… and in the end there is nothing that helps a programmatic configuration, apparently there is no support for the configuration programmatically, although I have done all this manually and programmatically to see what changes there are or what changes occur in the setting file, it turns out that… it doesn’t make much sense… I’m almost sure that even though setting is well configured , when opening docker-desktop, it reads the file but is not sending the parameter to //./pipe/dockerBackendV2 so that it is responsible for updating the hosts file… but it is not clear what is happening since I do not see errors notification. not a log or anything… I’m trying a complete uninstall with geek and then installing from 0.

Let’s summarize what I’m doing, I’ve left C# aside to evaluate what makes docker desktop unstable, and this is the complete powershell script I’m deploying, with the changes I need in setting.json, the script works correctly, but once that docker-desktop opens after 30 seconds, it is completely unstable:

$ErrorActionPreference = 'Continue'
Set-Service -Name "com.docker.service" -StartupType Automatic -ErrorAction SilentlyContinue
Stop-Service -Name "com.docker.service" -ErrorAction SilentlyContinue
Get-Process 'Docker Desktop' -ErrorAction Ignore | Stop-Process -Force -ErrorAction Stop
Wait-Process -Name 'Docker Desktop' -ErrorAction SilentlyContinue
$settingsToUpdate = @{
    exposeDockerAPIOnTCP2375 = $true
    updateHostsFile = $true
    licenseTermsVersion = 2
    noWindowsContainers = $true
    runWinServiceInWslMode = $false
    useResourceSaver = $false
}

$settingsPath = "$env:APPDATA\Docker\settings.json"
$settingsContent = Get-Content $settingsPath -Raw 
$settingsObject  = $settingsContent | ConvertFrom-Json

$trackUpdates = 0
foreach ($update in $settingsToUpdate.GetEnumerator()) {
    if ($target = $settingsObject.psobject.Properties.Match($update.Key)) {
        if ($target.Value -ne $update.Value) {
            Add-Member -InputObject $settingsObject -MemberType NoteProperty -Name $update.Key -Value $update.Value -Force
            $trackUpdates++
        }
    }
}

if ($trackUpdates -eq 0) {
    Write-Host "No new settings applied"
} else {
    $settingsObject | ConvertTo-Json | Set-Content $settingsPath
    Write-Host "Settings updated and saved successfully"
}

Start-Service -Name "com.docker.service" -ErrorAction SilentlyContinue

while ((Get-Service -Name "com.docker.service").Status -ne "Running") {
    Write-Host (Get-Service -Name "com.docker.service").Status
    Start-Sleep -Seconds 1
}

if((Get-Service -Name "com.docker.service").Status -eq "Running"){
    Write-Host (Get-Service -Name "com.docker.service").Status
}

$dockerDesktopFilePath = $env:ProgramFiles | Join-Path -ChildPath 'Docker\Docker\Docker Desktop.exe'; Start-Process -FilePath $dockerDesktopFilePath

$ipcTimeout = New-TimeSpan -Seconds 20
$waitUntil = [datetime]::Now.Add($ipcTimeout)
$pipeOpen = $false
Write-Host 'Probing docker engine I/O pipe'
do {
  Start-Sleep -Milliseconds 100
  $pipeOpen = Test-Path -LiteralPath \\.\pipe\docker_engine
} until ($pipeOpen -or ($waitUntil -le [datetime]::Now))

if (-not $pipeOpen) {
  Write-Warning "Failed to observe named IPC pipe docker_engine within timeout"
  return
}

$responseTimeout = New-TimeSpan -Seconds 5
$waitUntil = [datetime]::Now.Add($responseTimeout)

Write-Host 'Querying docker server info'
do {
  Start-Sleep -Milliseconds 500
  $dockerInfoOutput = docker info 2>&1 
  $dockerInfoSuccess = $?
} until ($dockerInfoSuccess -or ($waitUntil -le [datetime]::Now))

if (-not $dockerInfoSuccess) {
  Write-Warning "docker info failed within timeout"
  return
}

Write-Host 'Docker Desktop Is Runing Now'

I have no way of knowing whether you have only one or more local users or several local administrators on your computer or whether you are using an Active Domain account or something else. This is why I am asking you. This is also why I asked if you are using a user that is a member of the “docker-users” group.

It is also not correct that a normal administrator has the highest rights in Windows as you say. To perform some actions in Windows, it is not enough to have administrator rights. There is another privileged SYSTEM (LocalSystem) account in Windows. And the Docker service also runs under this account. Therefore, you can assume that it is not enough to start the service with administrator rights. Unfortunately, I can’t verify this because I can’t run your program, so it’s just a guess.

It was also just a guess that it could be due to your Docker version, because with this version there is the problem that the default settings are reset to the default values at startup when you use the settings management. But this is also just a guess and therefore I ask you if you have updated the version and could exclude this as a reason.

Let’s see friend, since I have been publishing data here about the related problem, I have been mentioning that I am obviously doing everything from my user, and that I execute everything with administrator privileges over another user… and that I have already confirmed that I have permissions and I’m in the groups… I don’t know how you want me to prove it to you… or why you assume that what I tell you is a lie… and you assume that some of it is not correct… when The problem reported no longer has anything to do with the script… but with the instability when the application opens… you are looking for a way to make or damage the direction in which my publication and this conversation are oriented What we have doesn’t help at all.

I see it differently. And I never said you were lying. I just wanted to explain to you that the service runs as a system user and that you probably only start it with administrator rights. I don’t see that you start the service with system rights and you didn’t say that either. That’s why I suggested trying restart-service docker with administrator rights in powershell. Did you try that?

Unfortunately you do not answer my questions and persist in the problem and therefore we do not get any further here. But this way I can’t rule out any assumptions. I can understand your frustration, but at least I’m trying to help you.

1 Like

I knew I couldn’t really help here, so I didn’t read the posts until before but I have done it now and I had no inmpression of anyone assuming a lie. I only saw useful recommendations and questions. Sometimes we need to ask more questions and we can’t just write all the questions in a single message when those come to our mind one by one. Sometimes we also ask a question which we already asked or which you think is not related to the issue or you feel you already answered it, because this is how we can eventually understand the problem better.

Considering that @bluenobly tries to help with something that is not officially supported in Docker Desktop, we can appriciate all the help. I also appriciate what you are doing @arcanisgk. Sharing information and questions about how Docker Desktop works in a public topic which could be interesting to other users, but I think it is understandable that the documentation doesn’t explain everything that can be used to programmatically manage Docker Desktop as it was not created for that. It was created for an interactive user to start it and use it for development and then stop it manually… Everything else can be interesting. I also try to play with it sometimes, although not outside but inside the VM, but that is not documented either, because it is not something users should do. Anything you do with Docker Desktop without interacting with the official GUI directly can’t be supported. I’m not saying it wouldn’t be great, but if you want it to be officially supported, you could ask it in the roadmap

If Docker Inc implemented an API that could be used from a code, that would make everything easier. Until that you can try to figure out how it works today, maybe you finish it, and tomorrow everything changes and you have to start again.

1 Like

This will be my last comment, I am not going to do ramdon tests without any support or without indicating what the expected result should be… it seems that this is a waste of time, what you ask or say is not rational.

  1. I’m doing what any user would do…
  2. If it does not have the minimum permission requirements then docker would not and should not start the docker-engine or at least there should be some notification saying that the requirements are not being met.

It is totally contradictory, stupid and irrational that the problem is one of permissions, with the level of progress that I have shown you; unless now they tell me that docker-desktop is garbage and that the software does anything but what is expected.

For God’s sake, opening an application by double-clicking on a shortcut should be the same as doing this in poweshell:

$dockerDesktopFilePath = $env:ProgramFiles | Join-Path -ChildPath 'Docker\Docker\Docker Desktop.exe'; Start-Process -FilePath $dockerDesktopFilePath

There is no way to justify this:

the steps are:
1 download, install docker desktop.
2 open it and do the docker survey.
3. restart the PC.
4 open the docker-desktop again.
5. Download and re-name the script from gist anywhere.
6. Open a terminal (powershell) with administrator privileges.
7. run the script from the open terminal and wait…
8. When it is executed, new settings will be applied to docker-desktop, the docker desktop will be closed and reopened.
9. Once opened, give it 1 minute and you will see that it becomes unstable…
10. discover the reason for the instability.

no you do not

If you use SilentlyContinue excessively, then you cannot expect this. If the processes produce relevant errors, you will not be able to see them. You should use "SilentlyContinue " wisly.

Permissions are one of the most common reasons why things don’t work. Excluding it as a problem is always a good idea. Even if it’s not the problem in the end. And it is a fact that normal users with administrator rights do not have the highest privileges in Windows.

But it’s not the same. For example, heuristics in AV solutions block Powershell commands. But normaly they do not block double-clicking on a shortcut of Docker.

So I wasted my time and tried things at random. But it’s better to say that I went through it step by step. That’s how I got results instead of wasting my time waiting for a solution. But I can understand you becaus it’s less stressful. :wink:

I tried the following code snippet and was able to reproduce the problem.

$ServiceName = "com.docker.service"
$ProcessName = "Docker Desktop"

$arrService = Get-Service -Name $ServiceName
if ($arrService.Status -ne 'Running') {
    Stop-Service -Name $ServiceName
}


if (Get-Process $ProcessName -ErrorAction SilentlyContinue) {  
  Stop-Process -Name $ProcessName
  Wait-Process -Name $ProcessName
}

In my case, the service would not be running, so the upper section of the code does not matter.
In the task manager I could see that 4 or respectively 6 processes are running in relation to Docker desktop. So that the Stop-Process command terminated the porcess “Docker Desktop” immediately.
But 2 processes “Docker Desktop Backend” and “Docker Desktop Extensions” are still running.
I started Docker Desktop by double-clicking on a shortcut and could then see that the Docker Engine was paused and the application was not running stably. I then terminated all processes and was able to open Docker Desktop again without any problems. From here on, you have to find out everything else for yourself.

On the other hand, run restart-service docker in Powershell for me is still the better solution.
Keep it simple, stupid (KISS)

So the problem is not how the application is opened, but how it is closed… it is not clear to me how many processes have to be closed when closing Docker-Desktop, it seems that the closing requires a wild-card: “Docker-Desktop*” or “Docker Desktop*” or “Docker” but this can cause processes that should be kept running to close.

On the other hand I found something curious last night when I did installation #64 from 0.

It turns out that if you have docker running and you go to the settings for the first time and check the option to update the hosts file, the updateHostsFile property is not the only one that changes; These other 2 also change: noWindowsContainers, runWinServiceInWslMode and screw up the general docker configuration, if you are using Windows 11/10 home, since these options should only be enabled if you are using the PRO version with Hiper-V.

So for this reason, if you programmatically activate updateHostsFile it will have no effect… unless you activate the other two… which will not have positive results either…

The thing is complicated and complex… it should be simpler to manage this through PowerShell…