well, you would have to docker attach to the container and the tool would have to be text mode. other than that, the same tools should work…
For testing I have a c program running inside my docker container. Basically it’s a for loop which runs forever and prints sth.
If I attach to the docker container from another terminal I get the same output.
How can I execute gdb on the running process then?
how do you do gdb normally on a running process… exactly the same
So I get the pid of the running process inside the container and run gdb.
then, I type attach PID
and I get this:
Could not attach to process. If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
gdb has to be installed and used INSIDE the container too, right?
Yes, first I run this command docker exec -t -i 0bf3c5863b0d /bin/bash
to access the container and then I run gdb.
or you could run docker attach container_id
i don’t know gdb, so I don’t know what that error means, but google search might help
Running docker attach container_id I see the output of the running process.
I tried to figure out what’s going on with gdb but still no luck…
gdb attach says
For a process id, you must have permission to send the process a signal,
and it must have the same effective uid as the debugger.
When using “attach” with a process id, the debugger finds the
program running in the process, looking first in the current working
directory, or (if not found there) using the source file search path
(see the “directory” command).
You can also use the “file” command
to specify the program, and to load its symbol table
so the process file needs to be debuggable (have symbols attached)… not just any executable
Yeah… I compiled the file with -g option and then gdb ./a.out but still have errors
and you are executing gdb from the folder with the program file in it?
Yes, they are both at the same folder.
ok, well that is all i know about gdb… hopefully someone else will chime in
I appreciate the help, thanks a lot.
If you can set the program to debug to STDOUT, then you can view it from docker logs.
The process is already running, I was wondering if there is a way to inject in a running process and see the memory state and the values of variables for example…
Why not just restart the container?
@ntwrkguru How about for the scenario that you cannot restart the container because you need to troubleshoot the process running in the container as-is?
I have yet to find a solution for debugging a running process in a running container. We have been trying something like
watch -n 1 'cat /proc/<PID>/task/<PID>/status' but I can’t find anything in there that points me to current syscalls and their output.
I am struggling with the same - where I need to troubleshoot a process running in a container as-is, restarting is not an option.
I have a python process in a docker container, which occasionally gets “stuck” and I try to figure out why.
The app is running by a dedicated app user, so I have logged in with root and then I have tried pyrasite within the container:
$ pip install pyrasite
$ echo 0 | tee /proc/sys/kernel/yama/ptrace_scope
$ pyrasite [PID] dump_stacks.py --verbose
b’ptrace: Operation not permitted.\nNo symbol table is loaded. Use the “file” command.\nNo symbol table is loaded. Use the “file” command.\nNo symbol table is loaded. Use the “file” command.\n’
I too am yet to find a solution for debugging a running process in a running container.
For now I just create a special, “debug” version of my container with the following flags to allow me to run gdb: