Can't get trivial autotest to run

rdzman · January 25, 2019, 5:45pm

I’m trying to get automated testing to work on a trivial example, as per …

… and I must be missing something obvious. I can’t seem to get the test to even run at all, though my image is being auto-built and pushed just fine.

I have the following 3 files in my source repo and I have followed the directions on the page linked above to turn on the Autotest option for “Internal and external pull requests”.

Dockerfile:

FROM debian:stable
WORKDIR /workspace
COPY run_docker_tests.sh /usr/local/bin

docker-compose.test.yml

sut:
  build: .
  command: run_docker_tests.sh

run_docker_tests.sh

#!/bin/bash
echo "Running run_docker_tests.sh FAILED"
exit 1

Pushing a commit to the source repo triggers a build on Docker Hub that outputs the following and then reports SUCCESS.

Building in Docker Cloud's infrastructure...
Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '192.30.253.113' to the list of known hosts.
Switched to a new branch 'docker-testing'
Pulling cache layers for index.docker.io/rdzman/test:test...
Done!
KernelVersion: 4.4.0-1060-aws
Components: [{u'Version': u'18.03.1-ee-3', u'Name': u'Engine', u'Details': {u'KernelVersion': u'4.4.0-1060-aws', u'Os': u'linux', u'BuildTime': u'2018-08-30T18:42:30.000000000+00:00', u'ApiVersion': u'1.37', u'MinAPIVersion': u'1.12', u'GitCommit': u'b9a5c95', u'Arch': u'amd64', u'Experimental': u'false', u'GoVersion': u'go1.10.2'}}]
Arch: amd64
BuildTime: 2018-08-30T18:42:30.000000000+00:00
ApiVersion: 1.37
Platform: {u'Name': u''}
Version: 18.03.1-ee-3
MinAPIVersion: 1.12
GitCommit: b9a5c95
Os: linux
GoVersion: go1.10.2
Starting build of index.docker.io/rdzman/test:test...
Step 1/3 : FROM debian:stable
---> 39bd7db3d7b5
Step 2/3 : WORKDIR /workspace
---> Using cache
---> 06954879f178
Step 3/3 : COPY run_docker_tests.sh /usr/local/bin
---> 75316af1abec
Successfully built 75316af1abec
Successfully tagged rdzman/test:test
Pushing index.docker.io/rdzman/test:test...
Done!
Build finished

But locally, I get the following …

$ docker run -it rdzman/test:test run_docker_tests.sh
Running run_docker_tests.sh FAILED
$ echo $?
1

What am I missing?

rdzman · January 31, 2019, 6:19pm

Alternatively, could someone point me to a complete “Hello World”-level example of a working auto test setup, ideally with a failing test that would prevent the pushing of the auto build?

Thanks in advance.

gforghetti · February 3, 2019, 12:35am

Try this to debug.

Run a container from your image and bring up a bash shell.

docker container run -it rdzman/test:test bash
Once inside the container’s bash prompt run your script.

run_docker_test.sh

rdzman · February 3, 2019, 3:41am

Thanks. But that gives exactly the same result. The script runs fine, indicating that it FAILED and it returns a non-zero (1) exit code.

The issue is that for some reason it seems the auto-test is not being run at all as part of the auto-build process.

gforghetti · February 3, 2019, 2:51pm

Ok, I’m starting to understand somewhat now. You are expecting the run_docker_tests.sh script to fail in your test, and it does locally when you run a container from the rdzman/test image. But when you push the rdzman/test docker image to Docker Hub the “auto build test” is not being triggered?

gforghetti · February 3, 2019, 7:58pm

Hi.

You asked for a complete “Hello World”-level example of a working auto test setup.

Let it be said, let it be written, let it be done.

_

gforghetti · February 3, 2019, 8:28pm

I just figured out why your automated test is not being triggered.
Your docker compose yml file name is incorrect.
Rename your docker-compose.test.yml file to docker-compose-test.yml and push that change up to your GitHub or Bitbucket repository. That should trigger the Automated Build and Test on your Docker Hub image repository.

rdzman · February 4, 2019, 3:07pm

Exactly! You got it. That’s exactly what I’m trying to do.

rdzman · February 4, 2019, 3:14pm

OK, I tried renaming docker-compose.test.yml to docker-compose-test.yml, but it doesn’t seem to have made any difference. The test still isn’t running. I will point out that documentation on Automated repository tests page says docker-compose.test.yml, not docker-compose-test.yml. But neither appears to be working for me.

I can’t see the build logs for your HelloWorld example here (to confirm that it’s running the auto test), but I guess I’ll try follow everything exactly to see if I can get it to work on my accounts.

gforghetti · February 4, 2019, 3:26pm

Well, I just renamed my docker-compose-test.yml file to docker-compose.test.yml and pushed that change up.
The build kicked off and the test ran!

35%20AM

rdzman · February 4, 2019, 3:31pm

I just forked your helloworld and set up my own auto build on Docker Hub and I can confirm that it does run the tests.

So, the mystery remains … what is wrong with the setup of mine that does not work?

gforghetti · February 4, 2019, 3:34pm

Can you post more details on your docker-compose.test.yml file and a screen capture of the Build Logs (similar to my screen capture).

rdzman · February 4, 2019, 3:36pm

The full content of my Dockerfile, docker-compose.test.yml and build log are in my initial post.

gforghetti · February 4, 2019, 3:41pm

Try this in your docker-compose.test.yml file. I’m shooting in the dark here myself.

version: '3.3'

services:
 sut:
   build: .
   command: run_docker_tests.sh

rdzman · February 4, 2019, 3:58pm

I had the same thought, but it doesn’t seem to have made a difference. I’m still getting the following. Is it something with my Dockerfile?

Building in Docker Cloud's infrastructure...
Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '192.30.253.113' to the list of known hosts.
Switched to a new branch 'docker-testing'
Pulling cache layers for index.docker.io/rdzman/test:test...
Done!
KernelVersion: 4.4.0-1060-aws
Components: [{u'Version': u'18.03.1-ee-3', u'Name': u'Engine', u'Details': {u'KernelVersion': u'4.4.0-1060-aws', u'Os': u'linux', u'BuildTime': u'2018-08-30T18:42:30.000000000+00:00', u'ApiVersion': u'1.37', u'MinAPIVersion': u'1.12', u'GitCommit': u'b9a5c95', u'Arch': u'amd64', u'Experimental': u'false', u'GoVersion': u'go1.10.2'}}]
Arch: amd64
BuildTime: 2018-08-30T18:42:30.000000000+00:00
ApiVersion: 1.37
Platform: {u'Name': u''}
Version: 18.03.1-ee-3
MinAPIVersion: 1.12
GitCommit: b9a5c95
Os: linux
GoVersion: go1.10.2
Starting build of index.docker.io/rdzman/test:test...
Step 1/3 : FROM debian:stable
---> 39bd7db3d7b5
Step 2/3 : WORKDIR /workspace
---> Using cache
---> 36f2877ebb1f
Step 3/3 : COPY run_docker_tests.sh /usr/local/bin
---> Using cache
---> ddcdf40b4b98
Successfully built ddcdf40b4b98
Successfully tagged rdzman/test:test
Pushing index.docker.io/rdzman/test:test...
Done!
Build finished

gforghetti · February 4, 2019, 4:42pm

I don’t see a CMD or ENTRYPOINT in the build log from your Dockerfile. That might be the problem. Do you have a CMD or ENTRYPOINT in your Dockerfile?

rdzman · February 4, 2019, 5:54pm

No, that wasn’t the issue I finally found it after taking my fork of your working helloworld repo and modifying things step-by-step until it was identical to mine and still working, while my original was still not working. At this point, I began to question my sanity.

The culprit … an invisible space at the end of my docker-compose.test.yml filename!!!

I can’t believe I didn’t think to check for something like that immediately. But thanks, Gary, for your time and the working example. I was missing the version and services lines in my docker-compose.test.yml file and having something that worked was what finally allowed me to track down the culprit.

So, thanks a ton!

gforghetti · February 4, 2019, 5:55pm

Awesome! Glad you got it working.