Share and learn in the Docker community.
Part of the ros2 rust installation process is to
mkdir -p workspace/src && cd workspace
git clone https://github.com/ros2-rust/ros2_rust.git src/ros2_rust
vcs import src < src/ros2_rust/ros2_rust_humble.repos
. /opt/ros/humble/setup.sh
colcon build
As you can see from the project docker file, I’m pretty far along with my ros2 development container. The point where I need to go into the workspace to do the final installation steps. Do you know what to do?
What’s the question? You have a Dockerfile and you know which commands to run, why not just RUN them?
Note: to reduce traffic, time and size, make sure to do a shallow git clone:
git clone --depth 1 <repository-url>
I didn’t know, it was that simple
because… its not!
That’s the error message I got:
=> ERROR [14/14] RUN cd /ros_ws && git clone https://github.com/ros2-rust/ros2_rust.git src/ros2_rust && 0.4s
------
> [14/14] RUN cd /ros_ws && git clone https://github.com/ros2-rust/ros2_rust.git src/ros2_rust && vcs import
src < src/ros2_rust/ros2_rust_humble.repos && . /opt/ros/humble/setup.sh && colcon build:
0.312 /bin/sh: 1: cd: can't cd to /ros_ws
------
Dockerfile:53
--------------------
52 | VOLUME /ros_ws/src
53 | >>> RUN cd /ros_ws && git \
54 | >>> clone https://github.com/ros2-rust/ros2_rust.git src/ros2_rust && \
55 | >>> vcs import src < src/ros2_rust/ros2_rust_humble.repos && \
56 | >>> . /opt/ros/humble/setup.sh && \
57 | >>> colcon build
58 |
--------------------
ERROR: failed to solve: process "/bin/sh -c cd /ros_ws && git clone https://github.com/ros2-rust/ros2_rust.git src/ro
s2_rust && vcs import src < src/ros2_rust/ros2_rust_humble.repos && . /opt/ros/humble/setup.sh && col
con build" did not complete successfully: exit code: 2
Indeed, but without knowing what you tried and what error message you got, it would have been hard to help.
The VOLUME is just a metadata in the Dockerfile until the container starts. You still need to create the folders in build time. It could happen by copying a file into the image using the COPY instruction or using mkdir in the run instruction.
I also don’t recommend using the VOLUME instruction. Many image maintainers already stopped using it as volumes defined in a Dockerfile can’t be undone and users are forced to override the mount point if they don’t want many unused anonymous volumes.
So to address a few points in your comment. I use the corresponding Dockerfile as part of my gitlab project. Here I need these VOLUME commands so that the built and started container is connected to the correct folders in my host environment. I edit my created packages with Neovim, which exists on my operating system and I need the container so that I can build the packages correctly.
Point for you. So here is the dockerfile.
ARG ROS_DISTRO=humble
ARG DEBIAN_FRONTEND=noninteractive
FROM ros:$ROS_DISTRO
ENV DISPLAY $DISPLAY
# install ros2 packages
RUN apt-get update && apt-get install -y --no-install-recommends \
ros-humble-desktop-full=0.10.0-1* \
&& rm -rf /var/lib/apt/lists/*
RUN apt update -y && apt upgrade -y
RUN apt install software-properties-common -y
RUN add-apt-repository universe
RUN apt install -y --no-install-recommends \
curl \
fuse \
git \
libfuse2 \
libclang-dev \
pip \
python3-pip \
python3-vcstool \
python3-colcon-common-extensions \
ros-humble-navigation2 \
ros-humble-nav2-bringup \
ros-humble-turtlebot3-gazebo \
ros-humble-robot-localization \
ros-humble-ros2bag \
ros-humble-rosbag2-storage-default-plugins \
tmux \
wget
RUN pip install ros2-numpy \
colcon-meson \
-U colcon-gradle \
--upgrade pytest \
git+https://github.com/colcon/colcon-cargo.git \
git+https://github.com/colcon/colcon-ros-cargo.git
RUN pip3 install opengen
# get Rust
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH=/root/.cargo/bin:$PATH
RUN echo 'source $HOME/.cargo/env' >> $HOME/.bashrc
RUN cargo install cargo-outdated evcxr_repl cargo-ament-build
RUN echo "eval '$(register-python-argcomplete3 ros2)'" >> $HOME/.bashrc
RUN echo "eval '$(register-python-argcomplete3 colcon)'" >> $HOME/.bashrc
VOLUME /ros_ws
VOLUME /microros_ws
VOLUME /ros_debug_ws
VOLUME /ros_ws/src
RUN cd /ros_ws && git \
clone https://github.com/ros2-rust/ros2_rust.git src/ros2_rust && \
vcs import src < src/ros2_rust/ros2_rust_humble.repos && \
. /opt/ros/humble/setup.sh && \
colcon build
Parts of the ros2 installation process take place directly in the created folders. I think that’s stupid too. Nevertheless, I have to intercept this somehow. Do you happen to know what I have to do?
I’ve tried to find out what you want from me. It does not work. Even if I use only the COPY commands, I get errors for the COPY commands.
ARG ROS_DISTRO=humble
ARG DEBIAN_FRONTEND=noninteractive
FROM ros:$ROS_DISTRO
ENV DISPLAY $DISPLAY
# install ros2 packages
RUN apt-get update && apt-get install -y --no-install-recommends \
ros-humble-desktop-full=0.10.0-1* \
&& rm -rf /var/lib/apt/lists/*
RUN apt update -y && apt upgrade -y
RUN apt install software-properties-common -y
RUN add-apt-repository universe
RUN apt install -y --no-install-recommends \
curl \
fuse \
git \
libfuse2 \
libclang-dev \
pip \
python3-pip \
python3-vcstool \
python3-colcon-common-extensions \
ros-humble-navigation2 \
ros-humble-nav2-bringup \
ros-humble-turtlebot3-gazebo \
ros-humble-robot-localization \
ros-humble-ros2bag \
ros-humble-rosbag2-storage-default-plugins \
tmux \
wget
RUN pip install ros2-numpy \
colcon-meson \
-U colcon-gradle \
--upgrade pytest \
git+https://github.com/colcon/colcon-cargo.git \
git+https://github.com/colcon/colcon-ros-cargo.git
RUN pip3 install opengen
# get Rust
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH=/root/.cargo/bin:$PATH
RUN echo 'source $HOME/.cargo/env' >> $HOME/.bashrc
RUN cargo install cargo-outdated evcxr_repl cargo-ament-build
RUN echo "eval '$(register-python-argcomplete3 ros2)'" >> $HOME/.bashrc
RUN echo "eval '$(register-python-argcomplete3 colcon)'" >> $HOME/.bashrc
VOLUME /ros_ws
VOLUME /microros_ws
VOLUME /ros_debug_ws
VOLUME /ros_ws/src
COPY $HOME/ros_ws /ros_ws
COPY $HOME/microros_ws /microros_ws
COPY $HOME/ros_debug_ws /ros_debug_ws
COPY $HOME/ros_ws/src /ros_ws/src
RUN cd /ros_ws && git \
clone https://github.com/ros2-rust/ros2_rust.git src/ros2_rust && \
vcs import src < src/ros2_rust/ros2_rust_humble.repos && \
. /opt/ros/humble/setup.sh && \
colcon build
Defining the volume in the Dockerfile is not necessary, and if you want to share your image, not even recommended. You can still define the volume paths when you start the contanier and have the same result. Except, you will have more control over where the files are. Since you mentioned editing the files, I have a blogpost which was inspired by people who wanted to edit files on volumes nd they asked where the volumes were on the filesystem. You should not edit anything under the docker data root where the default anonymous volumes and even named volumes are, but there are solutions: Everything about Docker volumes - DEV Community
That was just a sidenote, not the answer to your main question.
It would help, if you could share the error messages or we just guess a lot.
So my guess is that you got an error that the source path didn’t exist. Or the variable didn’t exist. The source path of the COPY instruction is always relative in your build context. Even if you use a variable in the path, that must be defined in ARG or ENV. It will not use $HOME existing on your host machine. Even if the variable is defined, if it is absolute, that will not work. Just imagine if I could create a Dockerfile and ask you to build the image and the docker build copied your home folder with all your secrets into the image and send it to somewhere. Everythigng has to be in the build context.
I see you changed the RUN instruction to use mkdir. If you don’t have anything to copy, that is the easiest and the right way.
That’s the error message:
=> ERROR [13/17] COPY /ros2_ws /ros_ws 0.0s
=> ERROR [14/17] COPY /microros_ws /microros_ws 0.0s
=> ERROR [15/17] COPY /ros2_debug_ws /ros_debug_ws 0.0s
=> ERROR [16/17] COPY /ros2_ws/src /ros_ws/src 0.0s
------
> [13/17] COPY /ros2_ws /ros_ws:
------
------
> [14/17] COPY /microros_ws /microros_ws:
------
------
> [15/17] COPY /ros2_debug_ws /ros_debug_ws:
------
------
> [16/17] COPY /ros2_ws/src /ros_ws/src:
------
Dockerfile:53
--------------------
51 | COPY $HOME/ros2_ws /ros_ws
52 | COPY $HOME/microros_ws /microros_ws
53 | >>> COPY $HOME/ros2_debug_ws /ros_debug_ws
54 | COPY $HOME/ros2_ws/src /ros_ws/src
55 | RUN mkdir -p /ros_ws/src && cd /ros_ws && colcon build
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref d65a38ee-ccaa-48f1-876c-04baee367c9c::iqda4crj1skr462txywws9u4r: "/ros2_debug_ws": not found
that’s the dockerfile version:
ARG ROS_DISTRO=humble
ARG DEBIAN_FRONTEND=noninteractive
FROM ros:$ROS_DISTRO
ENV DISPLAY $DISPLAY
# install ros2 packages
RUN apt-get update && apt-get install -y --no-install-recommends \
ros-humble-desktop-full=0.10.0-1* \
&& rm -rf /var/lib/apt/lists/*
RUN apt update -y && apt upgrade -y
RUN apt install software-properties-common -y
RUN add-apt-repository universe
RUN apt install -y --no-install-recommends \
curl \
fuse \
git \
libfuse2 \
libclang-dev \
pip \
python3-pip \
python3-vcstool \
python3-colcon-common-extensions \
ros-humble-navigation2 \
ros-humble-nav2-bringup \
ros-humble-turtlebot3-gazebo \
ros-humble-robot-localization \
ros-humble-ros2bag \
ros-humble-rosbag2-storage-default-plugins \
tmux \
wget
RUN pip install ros2-numpy \
colcon-meson \
-U colcon-gradle \
--upgrade pytest \
git+https://github.com/colcon/colcon-cargo.git \
git+https://github.com/colcon/colcon-ros-cargo.git
RUN pip3 install opengen
# get Rust
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH=/root/.cargo/bin:$PATH
RUN echo 'source $HOME/.cargo/env; eval "$(register-python-argcomplete3 ros2)"; eval "$(register-python-argcomplete3 colcon)"' >> $HOME/.bashrc
RUN echo "alias rosSource='source instal/setup.bash'"
RUN cargo install cargo-outdated \
evcxr_repl \
cargo-ament-build
COPY $HOME/ros2_ws /ros_ws
COPY $HOME/microros_ws /microros_ws
COPY $HOME/ros2_debug_ws /ros_debug_ws
COPY $HOME/ros2_ws/src /ros_ws/src
RUN mkdir -p /ros_ws/src && cd /ros_ws && colcon build
So what you’re telling me is, that I don’t even need the volume command inside the dockerfile, right?
So at the moment I’m running the installation of the ros2 environment in several steps, because running the installation steps completely in the Dockerfile didn’t work. First, the corresponding repositories have to be loaded into one of the created folders using my buildscript. Then I start the container and manually run colcon build; source install/setup.bash which sets up the ros2 environment properly. If I install everything in the Dockerfile, wouldn’t I have the problem that part of the content of the created folders exists in my host system and the other part in the container? Is that even possible?
Sorry, I’m not sure what you mean. If you describe the whole installation process in the Dockerfile, everything will be on the filesystem layers of the final image. Then when you start the container from the image, depending on where you mounted volumes, the content of some folders are copied to the host and the folder on the host is mounted into the container at exactly where the original files were copied from.
Also important to note that everything is on your host system as the container is mainly isolation. A process can’t see the whole host, but the process can be seen from outside the container. Including the files. You just mount some files into the isolated environment so the process can see that too.
Yes. You can define the volumes in a compose yaml file or as arguments of the docker run command.
That means ros2_debug_ws can’t be found on the host and $HOME is just ignored as an empty variable.
This is the solution that has been working for a few days now. However, it has several shortcomings.
COPY' command seems to be more tidy than the volume’ command. However, it seems that I don’t need the volume commands anymore.ARG ROS_DISTRO="humble"
ARG DEBIAN_FRONTEND=noninteractive
FROM ros:${ROS_DISTRO}
ENV DISPLAY $DISPLAY
# install ros2 packages
RUN apt update && apt install -y --quiet --no-install-recommends \
ros-${ROS_DISTRO}-desktop-full=0.10.0-1* \
apt update -y && apt upgrade -y && \
apt install -y --quiet --no-install-recommends \
software-properties-common \
&& add-apt-repository universe; \
apt update -y && apt upgrade -y && \
apt install -y --quiet --no-install-recommends \
bzip2 \
build-essential \
curl \
git \
gnupg2 \
libclang-dev \
pip \
python3-pip \
python3-vcstool \
python3-typeguard \
ros-${ROS_DISTRO}-generate-parameter-library \
ros-${ROS_DISTRO}-generate-parameter-library-py \
ros-${ROS_DISTRO}-navigation2 \
ros-${ROS_DISTRO}-nav2-bringup \
ros-${ROS_DISTRO}-parameter-traits \
ros-${ROS_DISTRO}-robot-localization \
ros-${ROS_DISTRO}-ros2bag \
ros-${ROS_DISTRO}-rosbag2-storage-default-plugins \
ros-${ROS_DISTRO}-ros2-control \
ros-${ROS_DISTRO}-ros2-controllers \
ros-${ROS_DISTRO}-rsl \
ros-${ROS_DISTRO}-turtlebot3-gazebo \
ros-${ROS_DISTRO}-test-interface-files \
wget && rm -rf /var/lib/apt/lists/* && \
apt autoclean -y && \
pip install ros2-numpy \
colcon-meson \
-U colcon-gradle \
--upgrade pytest \
pypozyx \
requests \
git+https://github.com/colcon/colcon-common-extensions.git \
git+https://github.com/colcon/colcon-cargo.git \
git+https://github.com/colcon/colcon-ros-cargo.git; \
pip3 install opengen; \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain 1.74.0 -y; \
echo 'source $HOME/.cargo/env; eval "$(register-python-argcomplete3 ros2)"' >> $HOME/.bashrc; \
echo 'eval "$(register-python-argcomplete3 colcon)"' >> $HOME/.bashrc && \
echo "alias rosSource='source ${HOME}/.bashrc'" >> $HOME/.bashrc; \
echo "alias colconBuild='colcon build --continue-on-error --build-base ros_ws/build --install-base ros_ws/install --base-paths /ros_default /ros_ws /ros_debug_ws /microros_ws'" >> $HOME/.bashrc;\
echo "source /opt/ros/humble/setup.bash" >> $HOME/.bashrc;\
echo "source /ros_ws/install/setup.bash" >> $HOME/.bashrc; \
echo "source /ros_debug_ws/install/setup.bash" >> $HOME/.bashrc; \
echo "source /microros_ws/install/setup.bash" >> $HOME/.bashrc;
ENV PATH=/root/.cargo/bin:$PATH
RUN cargo install --debug --no-track --verbose \
cargo-outdated cargo-ament-build;\
mkdir -p /ros_ws/src /microros_ws/src /ros_debug_ws/src /ros_default/src; \
git clone https://github.com/ros2-rust/ros2_rust.git /ros_default/src/ros2_rust; \
git clone -b ${ROS_DISTRO} https://github.com/ros2/common_interfaces.git /ros_default/src/common_interfaces; \
git clone -b ${ROS_DISTRO} https://github.com/ros2/example_interfaces.git /ros_default/src/example_interfaces; \
git clone -b ${ROS_DISTRO} https://github.com/ros2/rcl_interfaces.git /ros_default/src/rcl_interfaces; \
git clone -b ${ROS_DISTRO} https://github.com/ros2/test_interface_files.git /ros_default/src/test_interface_files; \
git clone -b ${ROS_DISTRO} https://github.com/ros2/rosidl_defaults.git /ros_default/src/rosidl_defaults; \
git clone -b ${ROS_DISTRO} https://github.com/ros2/unique_identifier_msgs.git /ros_default/src/unique_identifier_msgs; \
git clone https://gitlab.com/ros21923912/lanelet_msgs.git /ros_default/src/lanelet_msgs;\
. /opt/ros/${ROS_DISTRO}/setup.sh && colcon build \
--build-base ros_ws/build \
--install-base ros_ws/install \
--base-paths /ros_default /ros_ws /ros_debug_ws /microros_ws