Auto test working for branch, but not tag

I have two repositories that auto-build and auto-test from corresponding GitHub repositories. One is essentially a simple “hello world” example from here, the other a real project.

In both cases, everything works as expected when the builds are triggered by a “branch” rule in the auto-build configuration. An image is auto-built. That image is cached and used to run auto tests (set up according to discussion here). If the tests are successful, the image is tagged and pushed.

However, when the exact same commit triggers an auto build via a GitHub tag, the build fails when attempting the auto-test. In my hello world example, the error is …

...
Successfully built de91c2b9ebba
Successfully tagged rdzman/test:0.5
Starting Test in docker-compose.test.yml...
Building sut
Couldn't connect to Docker daemon at http+docker://localhost - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
building docker-compose.test.yml (1)

My docker-compose.test.yml file, which I remind you works just fine for auto builds triggered by a “branch” rule, contains only:

version: '3'
services:
  sut:
    build: .
    image: rdzman/test:${CACHE_TAG}
    command: /bin/bash run_docker_tests.sh

And my auto-build configuration is:
[Tag] /^[0-9.]+/, {sourceref}, Dockerfile, /, Autobuild [on], Build Caching [on]

My other repository, which has a similar setup, fails at the same step when triggered by a GitHub tag, but with a different error:

...
Successfully built f6726b74f521
Successfully tagged matpower/matpower-desktop:7.0
Starting Test in docker-compose.test.yml...
sut uses an image, skipping
Creating network "b7rfinanvhkh5qhbjkk8hrq_default" with the default driver
no such image: matpower/matpower-desktop:: invalid reference format
starting "sut" service in docker-compose.test.yml (1)

I’m not sure if the cause of failure is different for the two cases, but would appreciate help getting the hello world case working correctly first.

So, any ideas what I need to do differently to get it to build and test correctly when triggered by a GitHub tag?

Still haven’t been able to figure this out. Help, please!?

I think the only way to fix that is overwriting test routine or fixing your env on post checkout. The env ist different for both triggers. As you already found out sometimes the CACHE_TAG env is empty.

More information for overwriting: https://docs.docker.com/docker-hub/builds/advanced/#override-build-test-or-push-commands

Thanks for the response, but I’m afraid I don’t quite understand. Are you saying the problem is that CACHE_TAG is empty?

And how might I fix that? Add a hooks/post_checkout file? With what content?

I really am only trying to get working the absolute most simple auto-test set-up I can think of. Something that works when triggered by a “branch” rule and a by a “tag” rule. That is, I want to (1) auto-build an image, (2) run a test script in that image, and (3) push the image if and only if the test script succeeds. Does this really require Advanced options for Autobuild and Autotest?

Thanks again for any pointers.

Are you saying the problem is that CACHE_TAG is empty?

yes

And how might I fix that? Add a hooks/post_checkout file? With what content?

There is no general answer. It depends on your build logic and tagging logic. There are some other environment variables for example SOURCE_BRANCH, GIT_SHA1, SOURCE_COMMIT, GIT_TAG, DOCKER_TAG and IMAGE_NAME.
I said ‘fixing your env on post checkout’ but that’s wrong. You need to fix your environment variables on every step. Checkout my test repository hooks (https://github.com/Hackebein/docker-null/tree/4457cf724b0fbc318c38ece5486ea985b51712c5/hooks). This is the way i fixed the environment variables for me.

Does this really require Advanced options for Autobuild and Autotest ?

According to my knowledge, yes

Thank you for providing these details.

I have read some good stuff here. Certainly price bookmarking for revisiting. I wonder how much attempt you set to create this type of excellent informative web site.
http://canadianstoreguide.com/

Thanks for your time @hackebein, but I’m confess I’m still lost as to what specifically I need to change to get a simple “hello world” example to work for the simplest of both GitHub branch and GitHub tag auto-build rules.

@gforghetti, I really appreciated the help you gave me by creating a trivial example here (and updating it here). That was super helpful, but I find it only works for “branch” rules. Do you know how it can be modified to work for “tag” rules too?

Once again, I’m looking for the most trivial example possible that …
(1) auto-builds an image,
(2) runs a test script in that image, and
(3) pushes the image if and only if the test script succeeds.
… for both “branch” and “tag” auto-build rules.