Our hosts are in 10.x.x.x network, and the DNS server IP is 10.0.0.2. Now when we added an overlay network in Docker swarm without setting the subnet explicitly, Docker assigned the same 10.0.0.0/255 subnet to the overlay network. When the container tried to resolve a name, it was not able to talk to the 10.0.0.2 DNS server because IP packets got routed to the overlay network.
This was fixed by defining the subnets for overlay networks explicitly in another address range.
It was this GitHub issue that shed the light.