Can't communicate with kafka container from within .net core console application container

I have .NET Core 1.1 console application that is set up in a container. On the same host I have a kafka container running. I have set both up to be on the same user defined network, but I can’t get the console application to talk to the kafka container.

to set up the console app (named kafkacoreproducer), I use:
docker run -it --network yaps --name kafkacoreproducer --rm kafkacoreproducer

to set up the kafka container I used “docker-compose up” (where the container name was kafka), and then added the container to the network with:
docker network connect yaps kafka

In my console application, when configuring the Confluent.Kafka client, I’ve tried everything I can think of to get them to talk to each other. I’ve tried countless IP address combinations (based on the reported IP addresses in the kafka container), and even tried just using the container name (kafka) directly in the C# code.

How can I access the kafka container in my C# console application container?


If it helps, here’s the C# code:

  static void Main(string[] args)
            //string brokerList = "";
            string brokerList = ""; // this one works when run on local machine (not in container)
            //string brokerList = "kafka";
            //string brokerList = Environment.GetEnvironmentVariable("KAFKASERVER_CONTAINER");
            //Console.WriteLine($"KAFKASERVER_CONTAINER = {brokerList}");
            string topicName = "testObjects";

            var config = new Dictionary<string, object> { { "bootstrap.servers", brokerList } };
            Console.WriteLine($"starting producer on {brokerList}");

            using (var producer = new Producer<Null, string>(config, null, new StringSerializer(Encoding.UTF8)))
                Console.WriteLine("starting producer loop");
                while (true)
                    KafkaMessage message = new KafkaMessage { ProductName = ProductSettings.ProductName, ProductCount = ProductSettings.ProductCount };
                    Console.WriteLine(String.Format("created message: name={0}, count={1}", message.ProductName,message.ProductCount));
                    string payload = JsonConvert.SerializeObject(message);

                    Console.WriteLine("async produce");
                    var deliveryReport = producer.ProduceAsync(topicName, null, payload);
                    deliveryReport.ContinueWith(task =>
                        Console.WriteLine($"Partition: {task.Result.Partition}, Offset: {task.Result.Offset}");

                    Console.WriteLine("sleeping 5 seconds");


Have you looked at this article? You should use the name of the service as the hostname to get a hold of Kafka.

Yes, I saw that, but must’ve missed some permutations.

I finally got it working by altering my docker run command to:
docker run -it --network yaps --name=kafkacoreproducer --link=kafka:kafkaserver --rm kafkacoreproducer

Even though all the articles I’m reading nowadays say to avoid using the legacy link mechanism, this article (and the fact that it worked) seems to suggest it’s still necessary in some fashion in user defined networks. Fine by me.

For completeness, I then used “kafkaserver” as the string in my C# code:
string brokerList = "kafkaserver";

I really appreciate the help. Thank you