How to forward packets between net namespaces?

Question

I have a lima VM with linux running Ubuntu 23.04 with kernel 6.2 and cgroupsv2. This VM is meant to serve as docker host. When I create docker network in this VM I get an interface in its own namespace:

$ sudo lsns -t net | grep dockerd
4026532237 net      12  3807 lima   unassigned      /proc/self/exe --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run --propagation=rslave /usr/bin/dockerd-rootless.sh
$ sudo nsenter -n -t 3807 ip a s br-c6a402bfbc91
4: br-c6a402bfbc91: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:a6:e8:5d:1c brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global br-c6a402bfbc91
       valid_lft forever preferred_lft forever
    inet6 fc00:f853:ccd:e793::1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::42:a6ff:fee8:5d1c/64 scope link
       valid_lft forever preferred_lft forever
    inet6 fe80::1/64 scope link
       valid_lft forever preferred_lft forever

I have already came to a point where I can ping from that VM containers on 172.18.0.0/16 network

$ ping 172.18.1.1
PING 172.18.1.1 (172.18.1.1) 56(84) bytes of data.
64 bytes from 172.18.1.1: icmp_seq=2 ttl=64 time=0.166 ms
64 bytes from 172.18.1.1: icmp_seq=4 ttl=64 time=0.177 ms

but I cannot curl anything there

$ curl -v 172.18.1.1
*   Trying 172.18.1.1:80...

$ docker ps --filter name=lima-control-plane
CONTAINER ID   IMAGE                  COMMAND                  CREATED             STATUS             PORTS                       NAMES
fc12c34dff80   kindest/node:v1.27.1   "/usr/local/bin/entr…"   About an hour ago   Up About an hour   127.0.0.1:63869->6443/tcp   lima-control-plane
$ docker exec -it lima-control-plane curl 172.18.1.1
{
  "message":"no Route matched with those values"
}

unless I use the aforementioned namespace:

$ sudo nsenter -n -t 3807 curl 172.18.1.1
{
  "message":"no Route matched with those values"
}

Probably there's something messed up with my iptables, but I can't figure out what:

$ sudo iptables -n -L -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
  348 22152 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
  348 22152 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
  348 22152 ACCEPT     all  --  lima0  *       192.168.105.1        172.18.0.0/16
    0     0 ACCEPT     all  --  *      *       172.18.0.0/16        192.168.105.1

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
  778 49492 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
  778 49492 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0
$ ip r
default via 192.168.5.3 dev eth0 proto dhcp src 192.168.5.15 metric 100
default via 192.168.105.1 dev lima0 proto dhcp src 192.168.105.4 metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.5.0/24 dev eth0 proto kernel scope link src 192.168.5.15 metric 100
192.168.5.3 dev eth0 proto dhcp scope link src 192.168.5.15 metric 100
192.168.105.0/24 dev lima0 proto kernel scope link src 192.168.105.4 metric 100
192.168.105.1 dev lima0 proto dhcp scope link src 192.168.105.4 metric 100

$ sudo nsenter -n -t 3807 iptables -n -L -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      br-c6a402bfbc91  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-c6a402bfbc91  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-c6a402bfbc91 !br-c6a402bfbc91  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-c6a402bfbc91 br-c6a402bfbc91  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  lima0  *       192.168.105.1        172.18.0.0/16
    0     0 ACCEPT     all  --  lima0  *       192.168.105.1        172.18.0.0/16
    0     0 ACCEPT     all  --  lima0  *       172.18.0.0/16        192.168.105.1
    0     0 ACCEPT     all  --  *      lima0   172.18.0.0/16        192.168.105.1
    0     0 ACCEPT     all  --  *      *       172.18.0.0/16        192.168.105.1

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  !br-c6a402bfbc91 br-c6a402bfbc91  0.0.0.0/0            172.18.0.2           tcp dpt:6443
    0     0 ACCEPT     tcp  --  !br-c6a402bfbc91 br-c6a402bfbc91  0.0.0.0/0            172.18.0.4           tcp dpt:5000

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-c6a402bfbc91 !br-c6a402bfbc91  0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      br-c6a402bfbc91  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0
$ sudo nsenter -n -t 3807 ip r
default via 10.0.2.2 dev tap0
10.0.2.0/24 dev tap0 proto kernel scope link src 10.0.2.100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.0.0/16 dev br-c6a402bfbc91 proto kernel scope link src 172.18.0.1

What's interesting is that sudo ip netns list doesn't return anything. Not sure how that's related to the namespaces return via lsns.

EDIT: Apparently having docker installed in rootless mode is exactly the reason for the above.

I don't understand why you're trying to use an other network namespace while it appears you're using the initial network namespace: the br-* interface is in the namespace where Docker is running. Wouldn't the simple ip a s br-c6a402bfbc91 show the same, without trying to change namespace? Or is that a Docker-in-Docker? — A.B
– A.B, Commented Jun 19, 2023 at 9:37
The easy method to use the container's namespace is with docker inspect: docker inspect -f '{{.State.Pid}}' => nsenter -t $(docker inspect -f '{{.State.Pid}}' lima-control-plane) --net -- ip addr — A.B
– A.B, Commented Jun 19, 2023 at 10:59
ip a s br-c6a402bfbc91 wouldn't show this interface because it's in a different namespace. I already scratched this setup and installed rootful docker (running as root). What I experienced is probably due to how github.com/rootless-containers/slirp4netns creates user and net namespaces but I didn't get to the bottom of this (what exactly should be done to achieve the above). — Patryk
– Patryk, Commented Jun 20, 2023 at 10:16

Stack Exchange Network

How to forward packets between net namespaces?

0

You must log in to answer this question.

Hot Network Questions

How to forward packets between net namespaces?

0

You must log in to answer this question.

Related

Hot Network Questions