通常所说的DNAT,SNAT等包的转发,都必须要Linux内核IP Forward打开生效才能进行,再比如OpenStack里租户私有网络到了网关然后进行转换也是一样的道理,这里就通过一个小例子,仿照L3 namespace来验证下IP Forward的使能,大致流程
1:虚拟机里带一个公网IP
2:创建一个network namespace
3:创建一对veth设备,将namespace和虚拟机想通,并且外面的作为gateway
4:iptables添加SNAT规则,将src ip修改为公网IP
5:验证IP Forward使能
首先是一个带公网的虚拟机
~# ip a 1: lo: mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 4: eth2: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether fa:16:3e:1f:51:fa brd ff:ff:ff:ff:ff:ff inet 10.20.30.182/24 brd 10.20.30.255 scope global eth2 inet6 fe80::f816:3eff:fe1f:51fa/64 scope link valid_lft forever preferred_lft forever
然后创建namespace,veth设备等等
~# ip netns add ns_1 ~# ip netns list ns_1 ~# ip link add veth3 type veth peer name veth2 ~# ip link set veth2 netns ns_1 ~# ip netns exec ns_1 ifconfig veth2 1.1.1.2 netmask 255.255.255.0 up ~# ip netns exec ns_1 ip a 8: lo: mtu 16436 qdisc noop state DOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 9: veth2: mtu 1500 qdisc pfifo_fast state DOWN qlen 1000 link/ether c6:b9:fc:3a:7b:e5 brd ff:ff:ff:ff:ff:ff inet 1.1.1.2/24 brd 1.1.1.255 scope global veth2 ~# ~# ifconfig veth3 1.1.1.3 netmask 255.255.255.0 up ~# ip a show veth3 10: veth3: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether fe:9e:e0:1e:90:8c brd ff:ff:ff:ff:ff:ff inet 1.1.1.3/24 brd 1.1.1.255 scope global veth3 inet6 fe80::fc9e:e0ff:fe1e:908c/64 scope link valid_lft forever preferred_lft forever
如此一来,namespace里的veth2和虚拟机里的veth3这对peer设备就OK了,不放心的话测试一下连通性
~# ip netns exec ns_1 ping 1.1.1.3 PING 1.1.1.3 (1.1.1.3) 56(84) bytes of data. 64 bytes from 1.1.1.3: icmp_req=1 ttl=64 time=0.042 ms 64 bytes from 1.1.1.3: icmp_req=2 ttl=64 time=0.015 ms 64 bytes from 1.1.1.3: icmp_req=3 ttl=64 time=0.034 ms
这两个veth设备就像网线直连一样,所以能够直接通,想要namespace里的包能够出去,得加一个网关才能出去,正好veth3和veth2是能够连通的,作为网关再合适不过了
~# ip netns exec ns_1 route add default gw 1.1.1.3 ~# ip netns exec ns_1 ip r default via 1.1.1.3 dev veth2 1.1.1.0/24 dev veth2 proto kernel scope link src 1.1.1.2
这样veth2还是没法访问到公网的,因为它毕竟是内网IP,只能到veth3,但是虚拟机本身就带了一个公网IP,它和外界的连通性不用考虑,那么直接通过iptables做一层SNAT,将1.1.1.2转换成公网IP,这样包就可以出去了
~# iptables -t nat -A POSTROUTING -s 1.1.1.2 -j SNAT --to-source 10.20.30.182 ~# iptables -t nat -S -P PREROUTING ACCEPT -P INPUT ACCEPT -P OUTPUT ACCEPT -P POSTROUTING ACCEPT -A POSTROUTING -s 1.1.1.2/32 -j SNAT --to-source 10.20.30.182
理论上没问题了,来试试
~# ip netns exec ns_1 ping 114.114.114.114 PING 114.114.114.114 (114.114.114.114) 56(84) bytes of data. ^C --- 114.114.114.114 ping statistics --- 61 packets transmitted, 0 received, 100% packet loss, time 59999ms
还是不行,看下包的走向
~# tcpdump -i veth3 icmp -en tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on veth3, link-type EN10MB (Ethernet), capture size 65535 bytes 21:38:26.897236 c6:b9:fc:3a:7b:e5 > fe:9e:e0:1e:90:8c, ethertype IPv4 (0x0800), length 98: 1.1.1.2 > 114.114.114.114: ICMP echo request, id 60162, seq 15, length 64 21:38:27.897209 c6:b9:fc:3a:7b:e5 > fe:9e:e0:1e:90:8c, ethertype IPv4 (0x0800), length 98: 1.1.1.2 > 114.114.114.114: ICMP echo request, id 60162, seq 16, length 64 21:38:28.897249 c6:b9:fc:3a:7b:e5 > fe:9e:e0:1e:90:8c, ethertype IPv4 (0x0800), length 98: 1.1.1.2 > 114.114.114.114: ICMP echo request, id 60162, seq 17, length 64 21:38:29.897207 c6:b9:fc:3a:7b:e5 > fe:9e:e0:1e:90:8c, ethertype IPv4 (0x0800), length 98: 1.1.1.2 > 114.114. ^C 4 packets captured 4 packets received by filter 0 packets dropped by kernel ~# ~# tcpdump -i eth2 icmp -en tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel
可见,包到了veth3,但是还没到eth2,也就是说,iptables规则虽然期望修改源IP,但是由于某种原因没达到目的,这里就是因为Linux内核默认是关闭IP Forward的,包无法转发到eth2上
~# cat /proc/sys/net/ipv4/ip_forward 0 ~# echo 1 > /proc/sys/net/ipv4/ip_forward ~# cat /proc/sys/net/ipv4/ip_forward 1
这样就大功告成
~# ip netns exec ns_1 ping 114.114.114.114 PING 114.114.114.114 (114.114.114.114) 56(84) bytes of data. 64 bytes from 114.114.114.114: icmp_req=1 ttl=74 time=55.0 ms 64 bytes from 114.114.114.114: icmp_req=2 ttl=95 time=6.86 ms 64 bytes from 114.114.114.114: icmp_req=3 ttl=66 time=6.82 m