How to instantiate '3rd party' docker container from 'my' fastapi docker

I have a 3rd party docker image that I can run from command line as follows
$ lean backtest 'strategy'
(after I have pulled the image using $ docker pull quantconnect/lean)
This image executes the ‘strategy’ (a python module), dumps results and exit

if I were to run all this without docker, then on my computer then I would simply start fastapi server
$ uvicorn routes:app --reload
and on receiving the request, I would perform subprocess run['lean', 'backtest', 'strategy'], cwd=base_dir, capture_output=True, text=True)

I want to embed this in my fastapi docker server (to be able to finally run this on Azure)

If I create a docker image of fastapi and run the fastapi container then it complains that docker not running.
So as I understand within this fastapi container two things should happen 1. docker should be running 2. quanconnect/lean image should have been downloaded

Any ideas how to implement this in a way that I can later move this setup from my machine to Azure?

Of course you can run Docker in Docker and you can also run the Docker CLI inside your container and connect to your host to instantiate a parallel container, if you mount the host Docker socket into your container. But that’s all more theoretical.

I would create a new image, based on theirs, and add a small http wrapper, that receives http requests and then executes the script.

Python seems already installed (link), so that would be the obvious choice.

@bluepuma77 , thank you for your reply.
I am a bit new to this so please pardon my stupid question.
my current docker looks something like (inspired from internet)

FROM python:3.10-slim as compiler
RUN pip install -Ur requirements.txt
FROM python:3.10-slim as runner
COPY --from=compiler /app/venv /app/venv
ENV PATH="/app/venv/bin:$PATH"
ENV LEAN_PATH="/app/venv/bin/lean"
COPY . /app/
CMD python -m uvicorn --app-dir ./  --reload --host --port 8000 routes:app

Are you saying something like

  1. do a git pull and then use their Dockerfile and insert my docker content? or
  2. in my Docker file change
    FROM python:3.10-slim as compiler → FROM quantconnect/lean:latest
    and keep rest as it is?

If you mean 2nd option then is it cool especially if my works as it does currently.

Thank you for your guidance

I would have decoupled, FROM lean and add a simple http server to execute command.

But if you use Python anyway, you can try FROM lean for your regular app, too.

ok, thank you, let me try option 2 first (looks easier)… I will let you know …
(I do want to understand how to decouple without complex option like ‘Docker in Docker’ or Docker CLI you mentioned earlier)

While changing from FROM python to FROM quantconnect/lean worked in compilation …
the running did not work and gave this error
% docker run --rm -it -p 8000:8000 sappv01 bash

Welcome to .NET 6.0!

Could not execute because the specified command or file was not found.
Possible reasons for this include:

  • You misspelled a built-in dotnet command.
  • You intended to execute a .NET program, but dotnet-QuantConnect.Lean.Launcher.dll does not exist.
  • You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH.

It uses .NET underlying and the entrypoint is mentioned in the Dockerfile as
WORKDIR /Lean/Launcher/bin/Debug
ENTRYPOINT [ "dotnet", "QuantConnect.Lean.Launcher.dll" ]

Can you elaborate on your statement:

I would have decoupled, FROM lean and add a simple http server to execute command.

A Dockerfile always uses entrypoint (mostly a script) and cmd (as args) in combination. So you need to figure out how to combine that.

@bluepuma77 this was a useful tip…thank you

I added line ENTRYPOINT in my Dockerfile to prevent the base image (quantconnect/lean) ENTRYPOINT. This ran the fastapi server.
However, when I reached the stage in program['lean', 'backtest', 'strategy'], cwd=base_dir, capture_output=True, text=True)
the FastApi server complained:

Error: Please make sure Docker is installed and running

i.e. it does not realize that this container is already being called from within a container.
So back to my original problem. How would I run from fastapi docker container another container

You noted the original entrypoint

ENTRYPOINT [ "dotnet", "QuantConnect.Lean.Launcher.dll" ]

Do you run this inside the container?

lean backtest 'strategy'

Or is it a script outside, running Docker?

On my MacOS, If I am running my fastapi server manually from command line then flow is
$ uvicorn --app-dir . routes:app --reload
and this server on receiving the request calls lean['lean', 'backtest', 'strategy']).
Since Docker Desktop is running on my machine, the subprocess executes successfully.
Note: this lean process is ephemeral and exists after each run.

Now, from my viewpoint I only wrapped my fastapi in docker with command
CMD ["uvicorn", "--app-dir", "./", "routes:app", "--reload"]
and start this fastapi server from command line as
docker run --rm -it -p 8000:8000 sappv01

So lean backtest strategy is running inside the container and now if I think about your question the original entry point is is never being called.

It seams lean is a command installed by pip, which starts a Docker container (doc).