Can't use command in compose yaml

Open Source Projects Compose
1

Problem description

I need to create some directories and manage some files when a container is instantiated, and not when the image is built. Consequently, I’ve put the commands to perform these tasks in the docker-compose.yaml and not in the Dockerfile.

The problem that I’m encountering is that I can’t get any command to run properly from within the docker-compose.yaml. I get weird errors like.

Unrecognized alias: 'c', it will have no effect when I run a command with /bin/bash -c

or

No such file or directory: /app/bash when I run a command with bash, i.e. without /bin

similarly, if I try to run a python script I get

No such file or directory: /app/python

here is my docker-compose.yaml:

version: "3.4"

services:

  app:
    build:
      context: ./
      target: project_build
    image: sig-project
    environment:
      PROJECT_NAME: "${PROJECT_NAME:-NullProject}"
    command: /bin/bash -c "init_project.sh"
    ports:
      - 7777:7777
    expose:
      - 7777
    volumes:
      - ./$PROJECT_NAME:/app/$PROJECT_NAME
      - .:/home/jovyan/work
    working_dir: /app
    entrypoint: "jupyter notebook --ip=0.0.0.0 --port=7777 --allow-root --no-browser"

Problem location

I’m not sure what the source of the problem is. It’s clear from the error messages that the shell is looking for command folders in the working dir, but I’m not sure why this is or how to change it.

Project version(s) affected

Docker desktop version: 4.11.0 (83626)
Engine version: 20.10.17

$ docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020
2

So basicly it is your entrypoint script… right? Is there a reason that you don’t make /absolut/path/to/init_project.sh the ENTRYPOINT in the Dockerfile?
Also make sure your script is location aware, because it will run in the context of your working_dir, which is /app.

3

Interesting idea, I hadn’t considered that. It could be a viable workaround.

It still irks me a bit that the command directive doesn’t work.

I’m wondering if it might be a version issue as I’m running v1 of docker compose on macos 10.14.

4

What is the final command you want to run in the container? This is what your current compose file would run:

jupyter notebook --ip=0.0.0.0 --port=7777 --allow-root --no-browser /bin/bash -c "init_project.sh"

I don’t run jupyterbook often, but it doesn’t seem right.

5

Thanks, yeah this is similar to what was previously suggested. I’ll give it a try.

I don’t have any issues with jupyter. The issue is only with the declaration in the command directive.

6

I missed the declaration for entrypoint.

If entrypoint is present, the command will be an argument to it (like you can see in @rimelek’s last post) - this is what the container executes during start. If entrypoint is not declared (including the ones in the hiararchy of base images!), then command alone declares what is executed during start.

Make your bash script the entrypoint, make sure it uses exec jupyter ... at the end of your script to actualy start jupyter and let it assume pid1.

7

Since you can’t run the container and you want to run jupyter notebook in the container, I would say that is pretty much an issue :slight_smile: If you search for the first error message

It comes from jupyter, because -c is not a valid option and jupyter calls it an alias. Since your command was appended to the entrypoint, the bash command became a parameter of jupyter with -c as well.

1 Like
8

@rimelek thanks.

I can spin up the container fine. The only issue is that the command directive doesn’t ever actually call the indicated command, but throws the errors shared.

The unrecognized ‘-c’ is from the bash -c “init_project.sh” command.

9

Why do you think that? Another attempt to explain:

command: /bin/bash -c "init_project.sh"
...
entrypoint: "jupyter notebook --ip=0.0.0.0 --port=7777 --allow-root --no-browser"

…will make Docker do this:

10

…or is it? I wanted to add a link to Understand how CMD and ENTRYPOINT interact to my post above. But then I realized that both command and entrypoint in the original post are using the shell form (not the exec form). So I guess entrypoint is prefixed with /bin/sh -c and command is ignored altogether? Like so (maybe without the quotes):

/bin/sh -c "jupyter notebook --ip=0.0.0.0 --port=7777 --allow-root --no-browser"

Regardless: creating an entrypoint script that does all magic (and making entrypoint use that, and no longer specifying command) is the way to go. Remember:

See the 2nd link I provided above, which also says:

The shell form prevents any CMD or run command line arguments from being used, but has the disadvantage that your ENTRYPOINT will be started as a subcommand of /bin/sh -c, which does not pass signals. This means that the executable will not be the container’s PID 1 - and will not receive Unix signals - so your executable will not receive a SIGTERM from docker stop <container>.

See also the note about exec in “Configure app as PID 1” in https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#entrypoint.

11

Though that would not explain the different errors that were seen for different command values. Oh well. It’s just not the way to go. :sunglasses:

12

@avbentem thanks for the thorough explanation. This helps a lot. I didn’t realize that entry point and command we’re concatenated together.

Regardless, I was getting the same error even when I tried without entry point defined in the compose file and only had the command declaration

I’ll compile everything into a shell script to run at start and see if that works better.

I still don’t get why the directories for the commands indicated can’t be found. It seems that Im missing something fundamental. Anywhere where I can read more about this?

13

Of course not, because that never runs. And the reason is what I wrote.

It was your intention, yes, but this is not what happens. Just run this:

docker container ls --no-trunc --all --format 'table {{ .Names }}\t{{ .Command }}'

You will see the actual command running in your containers. When I wrote the “Unrecognized alias” error is thrown by jupyter I didn’t just guess. First of all, I have never seen bash giving that error message, and second: Unrecognized alias: '--profile=xxx', it will probably have no effect. · Issue #309 · jupyter/notebook · GitHub

I can see that you asked the same on StackOverflow python - command in docker-compose.yaml throwing errors - Stack Overflow

and reply the same to people who are trying to help

I don’t have a problem spinning up the container

but you have problem. Correct me if I am wrong, but you are trying to start the container, the container starts and immediately stops. An incorrect combination of command and entrypoint doesn’t mean compose would recognize it without letting Docker to run the command. Your container will start, the command fails and the container stops.

I had the same thought, but I like to test before I suggest something unless I also say that “It is just a guess”. So I tested and it looks like there is no exec form and shell form in case of Docker compose. The two different syntaxes will just help you avoid incorrect quoting.

update:
If you think about it, using the docker client there is no way to use an exec form, but a command will always be concatenated to the entrypoint.
/end update

I shared the concatenated entrypoint and command hoping you would realize, but thanks to @avbentem for clarifying it :slight_smile:

If by “same error” you mean the “unrecognized alias”, I am pretty sure you have a default entrypoint in the Dockerfile or in the base image which will also run jupyter. I can’t say for sure without the Dockerfile, but I can’t think of any other explanation.

2 Likes
14

Makes sense, and great you tested, and I am sure you’re right (and otherwise different command values would not yield different errors, assuming we got all details). I’d not have taken that away from the Compose documentation:

entrypoint

entrypoint overrides the default entrypoint for the Docker image (i.e. ENTRYPOINT set by Dockerfile). Compose implementations MUST clear out any default command on the Docker image - both ENTRYPOINT and CMD instruction in the Dockerfile - when entrypoint is configured by a Compose file. If command is also set, it is used as parameter to entrypoint as a replacement for Docker image’s CMD

Above, the “as a replacement for Docker image’s CMD” would have fooled me into thinking the same rules for interaction would apply. :see_no_evil:

1 Like
15

@rimelek thanks for the extensive response.

First, the container starts and runs fine. I don’t have any problem running it or working in it.

Second, I get the error messages indicated in the OP even if I exclude the entry point directive with jupyter.

16

Yes, you’re right. I had the entry point defined in the Dockerfile. So you’re saying that in both setups command and emtrypoint are being concatenated.

If this is the case, how can you define multiple independent commands and an entry point? Seems like an odd way to parse the config.

17

Thanks @avbentem for sharing the documentation. Things are starting to make more sense.

18

The command below will show you the command each image in your local cache executes when a container based on it is started:

docker inspect $(docker image ls -q)  --format '{{if .RepoTags}}{{.RepoTags}}{{else}}{{.RepoDigests}}{{end}} ENTRYPOINT:{{.Config.Entrypoint}}, CMD:{{.Config.Cmd}}, container starts: {{if  .Config.Entrypoint}}{{join .Config.Entrypoint " "}}{{end}} {{if .Config.Cmd}}{{join .Config.Cmd " "}}{{end}}'

If you want to check a specific image, replace $(docker image ls -q) with the “image name:image tag” or “image id” (the quotes are optional) .

Based on that knowledge, you can get an idea of how your changes to entrypoint: and command: aftect the command executed when the container is started.

Complex operations are ment to be handled within an entrypoint script, and not as a chain of commands in entrypoint: or command:

1 Like