I forgot to include the httpd-foreground script I am using, which was copied from https://github.com/docker-library/httpd/tree/master/2.4:
#!/bin/sh
set -e
# Apache gets grumpy about PID files pre-existing
rm -f /usr/local/apache2/logs/httpd.pid
exec httpd -DFOREGROUND "$@"