I finally got it working. Here is the compose file:
version: '3.4'
services:
web:
image: dockercloud/hello-world
service_auth:
image: redmarker/nginx-basic-auth
ports:
- 8081:80
depends_on:
- web
environment:
HTPASSWD: 'foo:$$apr1$$odHl5EJN$$KbxMfo86Qdve2FH4owePn.'
FORWARD_PORT: 80
FORWARD_HOST: web
$ characters must be escaped with a second $ character in docker-compose.yml.