Docker rootless : access a container with the same IP in both the host and another container

I want to access a container using the same IP, on both my host and another container.

I have the following configuration :

version: "3"

services:
  nginx:
    image: nginx
    ports:
      - 8080:80
    networks:
      cust_network:
        ipv4_address: 192.168.27.100

  cmdline:
    image: ubuntu
    networks:
      - cust_network

networks:
  cust_network:
    driver: bridge
    ipam:
      config:
        - subnet: 192.168.27.0/24```

From cmdline I can access ngnix only on 192.168.27.100 whereas on my host, I can only access it trough 172.17.0.1:8080 

What can I do ?

Thanks