Cannot consume Kafka from a dockerized .Net Core application

Docker Desktop
1

I’m running locally on my windows Docker Desktop with linux containers.

I created a zookeeper container using the following command :

docker run -d --network kafka --name=MyZookeeper -p 2181:2181 -e ALLOW_ANONYMOUS_LOGIN=yes -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:5.5.0

And a Kafka containier using this command :

docker run -d --network kafka --name=MyKafka -p 9092:9092 -e ALLOW_PLAINTEXT_LISTENER=yes -e KAFKA_ZOOKEEPER_CONNECT=MyZookeeper:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 confluentinc/cp-kafka:5.5.0

I have a non dockerized .Net core app that is publishing messages to a specific topic in my Kafka.
The publishing process works fine and I have been able to verify that using Conduktor.

The problem is when I try to consume this data from a dockerized app that I also have running locally. Here’s the docker-compose.yml file content :

version: '1.0'

services:
  api:
    image: ${DOCKER_REGISTRY}cds.Test.api
    ports:
      - "64788:64788"
      - "64789:64789"
    build:
      context: .
      dockerfile: src/Api/Dockerfile.Debug
    environment:
      - ASPNETCORE_URLS=http://+:64788
      - ASPNETCORE_HTTP_PORT=64788
      - MANAGEMENT_HTTP_PORT=64789
      - ASPNETCORE_ENVIRONMENT=Local
networks:
  default:
    external:
      name: kafka

on my appsetting.local.json file I have :

"consumerOptions": {
    "BootstrapServers": "localhost:9092",
    "GroupId": "GroupId",
    "TopicName": "my_topic,
    "EnableAutoCommit": true,
    "AutoOffsetReset": "Latest",
    "MaxPollIntervalMs": 120000,
    "StatisticsIntervalMs": 1000
  }

And then using this config I build a kafka consumer. But the problem is whenever I launch my dockerized app I get the following error :

%3|1618587835.523|FAIL|rdkafka#consumer-1| [thrd:localhost:9094/bootstrap]: localhost:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused (after 0ms in state CONNECT, 66 identical error(s) suppressed)

This only happens when I try to consume locally but it works fine when I consume a remote server.

2

hi!
I have the same issues, you were able to solve it?
thanks.