虚拟机VMware Fusion，操作系统CentOS7.2，网络配置桥接
devstack搭建好单节点openstack之后，直接创建虚拟机
[lihui@openstack devstack]$ nova boot --flavor 1 --image 5291f14d-a20d-41c7-a809-87b0a02a2e40 --nic net-id=8f91ac0f-54e4-4bbb-834b-551301aaa0d0 --nic net-id=11d20d58-99e8-4688-b9ab-a35d7149bdd8 --key-name dev --admin-pass 000000 new-vm
+--------------------------------------+-----------------------------------------------------------------+
| Property | Value |
+--------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | new-vm |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-bki2qg27 |
| OS-EXT-SRV-ATTR:root_device_name | - |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | 000000 |
| config_drive | |
| created | 2017-07-16T15:18:00Z |
| description | - |
| flavor | m1.tiny (1) |
| hostId | |
| host_status | |
| id | 5941b42b-1b26-4cce-be1c-ec2e5632bbcd |
| image | cirros-0.3.5-x86_64-disk (5291f14d-a20d-41c7-a809-87b0a02a2e40) |
| key_name | dev |
| locked | False |
| metadata | {} |
| name | new-vm |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tags | [] |
| tenant_id | 32e905a2f692421da826ce67e11a3fec |
| updated | 2017-07-16T15:18:00Z |
| user_id | 8f40ab52e7174816a502128aa0c10991 |
+--------------------------------------+-----------------------------------------------------------------+
namespace里连通虚拟机，不通
[lihui@openstack devstack]$ sudo ip netns exec qrouter-70ad5580-33a1-4f77-b242-1c3d2b71af8c ping 10.0.0.9 PING 10.0.0.9 (10.0.0.9) 56(84) bytes of data.
看下安全组
[lihui@openstack devstack]$ nova interface-list 5941b42b-1b26-4cce-be1c-ec2e5632bbcd +------------+--------------------------------------+--------------------------------------+----------------------------------------------+-------------------+ | Port State | Port ID | Net ID | IP addresses | MAC Addr | +------------+--------------------------------------+--------------------------------------+----------------------------------------------+-------------------+ | ACTIVE | a08c3748-2069-43ae-ae86-e58264f540ff | 11d20d58-99e8-4688-b9ab-a35d7149bdd8 | 172.24.4.9,2001:db8::a | fa:16:3e:3c:61:a9 | | ACTIVE | cc545042-dd6e-40c2-8783-af7591ad440d | 8f91ac0f-54e4-4bbb-834b-551301aaa0d0 | 10.0.0.9,fd8d:4a3:aeb0:0:f816:3eff:fef1:25e3 | fa:16:3e:f1:25:e3 | +------------+--------------------------------------+--------------------------------------+----------------------------------------------+-------------------+
查找PORT对应安全组
[lihui@openstack ~]$ neutron port-show a08c3748-2069-43ae-ae86-e58264f540ff | grep security | port_security_enabled | True | | security_groups | a35a41e6-77c8-45a8-b9ba-0c16a24bc910 |
添加ingress放通规则
[lihui@openstack devstack]$ neutron security-group-rule-create --direction ingress --remote-ip-prefix 0.0.0.0/0 a35a41e6-77c8-45a8-b9ba-0c16a24bc910 Created a new security_group_rule: +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | created_at | 2017-07-16T15:21:11Z | | description | | | direction | ingress | | ethertype | IPv4 | | id | 169ac2f6-d9ef-4c59-a003-63980ecf39f0 | | port_range_max | | | port_range_min | | | project_id | 32e905a2f692421da826ce67e11a3fec | | protocol | | | remote_group_id | | | remote_ip_prefix | 0.0.0.0/0 | | revision_number | 1 | | security_group_id | a35a41e6-77c8-45a8-b9ba-0c16a24bc910 | | tenant_id | 32e905a2f692421da826ce67e11a3fec | | updated_at | 2017-07-16T15:21:11Z | +-------------------+--------------------------------------+
通了
[lihui@openstack devstack]$ sudo ip netns exec qrouter-70ad5580-33a1-4f77-b242-1c3d2b71af8c ping 10.0.0.9 PING 10.0.0.9 (10.0.0.9) 56(84) bytes of data. ▽ 64 bytes from 10.0.0.9: icmp_seq=1 ttl=64 time=1.72 ms 64 bytes from 10.0.0.9: icmp_seq=2 ttl=64 time=0.507 ms ^C --- 10.0.0.9 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1002ms
看下到public IP
[lihui@openstack devstack]$ ping 172.24.4.9 PING 172.24.4.9 (172.24.4.9) 56(84) bytes of data. From 172.24.4.1 icmp_seq=1 Destination Host Unreachable From 172.24.4.1 icmp_seq=2 Destination Host Unreachable From 172.24.4.1 icmp_seq=3 Destination Host Unreachable From 172.24.4.1 icmp_seq=4 Destination Host Unreachable ^C --- 172.24.4.9 ping statistics --- 5 packets transmitted, 0 received, +4 errors, 100% packet loss, time 4005ms pipe 4
看下qg
[lihui@openstack devstack]$ sudo ip netns exec qrouter-70ad5580-33a1-4f77-b242-1c3d2b71af8c ip a show qg-9452f9cc-51
15: qg-9452f9cc-51: mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:56:98:50 brd ff:ff:ff:ff:ff:ff
inet 172.24.4.2/24 brd 172.24.4.255 scope global qg-9452f9cc-51
valid_lft forever preferred_lft forever
inet6 2001:db8::9/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe56:9850/64 scope link
valid_lft forever preferred_lft forever
连通性无误
[lihui@openstack devstack]$ ping 172.24.4.2 PING 172.24.4.2 (172.24.4.2) 56(84) bytes of data. 64 bytes from 172.24.4.2: icmp_seq=1 ttl=64 time=0.292 ms 64 bytes from 172.24.4.2: icmp_seq=2 ttl=64 time=0.076 ms 64 bytes from 172.24.4.2: icmp_seq=3 ttl=64 time=0.054 ms 64 bytes from 172.24.4.2: icmp_seq=4 ttl=64 time=0.055 ms
确认qg在br-int上
[lihui@openstack ~]$ sudo ovs-ofctl show br-int | grep qg-9452f9cc-51 17(qg-9452f9cc-51): addr:00:00:00:00:b0:37
也就是vm-tap-qbr-namespace这个链路有问题
获取VNC地址登陆
[lihui@openstack ~]$ nova get-vnc-console 5941b42b-1b26-4cce-be1c-ec2e5632bbcd novnc +-------+-------------------------------------------------------------------------------------+ | Type | Url | +-------+-------------------------------------------------------------------------------------+ | novnc | http://192.168.100.31:6080/vnc_auto.html?token=4927a9c5-62ac-4ec4-a6c6-fbf53b25e546 | +-------+-------------------------------------------------------------------------------------+
宿主机无法打开，查看端口，未放开
[lihui@openstack ~]$ telnet 192.168.10.31 6080 Trying 192.168.10.31... ^C
放通6080端口
[lihui@openstack ~]$ sudo iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 6080 -j ACCEPT
[lihui@openstack ~]$ sudo lsof -i:6080
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
nova-novn 69421 lihui 3u IPv4 135635 0t0 TCP *:6080 (LISTEN)
[lihui@openstack ~]$ ip a | grep 100.31
inet 192.168.100.31/24 brd 192.168.100.255 scope global dynamic ens33
[lihui@openstack ~]$ sudo iptables -vnL --line-numbers | grep 6080
13 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:6080
终于通了
[lihui@openstack ~]$ telnet 192.168.100.31 6080 Trying 192.168.100.31... Connected to 192.168.100.31. Escape character is '^]'. ^C Connection closed by foreign host.
可虚拟机里是MINI版，没有desktop，而宿主机mac上依旧telnet不通，抓包，有去无回
[lihui@openstack ~]$ sudo tcpdump -i ens33 port 6080 -en tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ens33, link-type EN10MB (Ethernet), capture size 65535 bytes 00:21:42.328150 a4:5e:60:d6:4f:df > 00:0c:29:8c:4e:c7, ethertype IPv4 (0x0800), length 78: 192.168.100.11.55146 > 192.168.100.31.6080: Flags [S], seq 391903154, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS val 715508390 ecr 0,sackOK,eol], length 0 ^C 1 packet captured 3 packets received by filter 0 packets dropped by kernel
不清楚是不是vnc有问题，先放弃
VM到qg不通，查看br-int流表
[lihui@openstack ~]$ sudo ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4): cookie=0xd957818e88e64f11, duration=60.796s, table=0, n_packets=0, n_bytes=0, idle_age=60, priority=10,icmp6,in_port=19,icmp_type=136 actions=resubmit(,24) cookie=0xd957818e88e64f11, duration=60.738s, table=0, n_packets=0, n_bytes=0, idle_age=60, priority=10,icmp6,in_port=20,icmp_type=136 actions=resubmit(,24) cookie=0xd957818e88e64f11, duration=60.789s, table=0, n_packets=2, n_bytes=84, idle_age=5, priority=10,arp,in_port=19 actions=resubmit(,24) cookie=0xd957818e88e64f11, duration=60.733s, table=0, n_packets=0, n_bytes=0, idle_age=60, priority=10,arp,in_port=20 actions=resubmit(,24) cookie=0xd957818e88e64f11, duration=64.506s, table=0, n_packets=0, n_bytes=0, idle_age=64, priority=2,in_port=1 actions=drop cookie=0xd957818e88e64f11, duration=60.805s, table=0, n_packets=7, n_bytes=686, idle_age=5, priority=9,in_port=19 actions=resubmit(,25) cookie=0xd957818e88e64f11, duration=60.750s, table=0, n_packets=0, n_bytes=0, idle_age=60, priority=9,in_port=20 actions=resubmit(,25) cookie=0xd957818e88e64f11, duration=61.609s, table=0, n_packets=0, n_bytes=0, idle_age=61, priority=3,in_port=1,vlan_tci=0x0000/0x1fff actions=mod_vlan_vid:2,NORMAL cookie=0xd957818e88e64f11, duration=64.671s, table=0, n_packets=31, n_bytes=1770, idle_age=5, priority=0 actions=NORMAL cookie=0xd957818e88e64f11, duration=64.675s, table=23, n_packets=0, n_bytes=0, idle_age=64, priority=0 actions=drop cookie=0xd957818e88e64f11, duration=60.802s, table=24, n_packets=0, n_bytes=0, idle_age=60, priority=2,icmp6,in_port=19,icmp_type=136,nd_target=fe80::f816:3eff:fef1:25e3 actions=NORMAL cookie=0xd957818e88e64f11, duration=60.799s, table=24, n_packets=0, n_bytes=0, idle_age=60, priority=2,icmp6,in_port=19,icmp_type=136,nd_target=fd8d:4a3:aeb0:0:f816:3eff:fef1:25e3 actions=NORMAL cookie=0xd957818e88e64f11, duration=60.746s, table=24, n_packets=0, n_bytes=0, idle_age=60, priority=2,icmp6,in_port=20,icmp_type=136,nd_target=fe80::f816:3eff:fe3c:61a9 actions=NORMAL cookie=0xd957818e88e64f11, duration=60.741s, table=24, n_packets=0, n_bytes=0, idle_age=60, priority=2,icmp6,in_port=20,icmp_type=136,nd_target=2001:db8::a actions=NORMAL cookie=0xd957818e88e64f11, duration=60.792s, table=24, n_packets=2, n_bytes=84, idle_age=5, priority=2,arp,in_port=19,arp_spa=10.0.0.9 actions=resubmit(,25) cookie=0xd957818e88e64f11, duration=60.735s, table=24, n_packets=0, n_bytes=0, idle_age=60, priority=2,arp,in_port=20,arp_spa=172.24.4.9 actions=resubmit(,25) cookie=0xd957818e88e64f11, duration=64.668s, table=24, n_packets=0, n_bytes=0, idle_age=64, priority=0 actions=drop cookie=0xd957818e88e64f11, duration=60.821s, table=25, n_packets=9, n_bytes=770, idle_age=5, priority=2,in_port=19,dl_src=fa:16:3e:f1:25:e3 actions=NORMAL cookie=0xd957818e88e64f11, duration=60.755s, table=25, n_packets=0, n_bytes=0, idle_age=60, priority=2,in_port=20,dl_src=fa:16:3e:3c:61:a9 actions=NORMAL [lihui@openstack ~]$
查看port 20是什么口
[lihui@openstack ~]$ sudo ovs-ofctl show br-int | grep "20("
20(qvoa08c3748-20): addr:56:0f:95:11:2f:d6
table 24和25，看样子ARP都没成功
[lihui@openstack ~]$ sudo ip netns exec qrouter-70ad5580-33a1-4f77-b242-1c3d2b71af8c arp -a ? (172.24.4.9) at on qg-9452f9cc-51 ? (10.0.0.2) at fa:16:3e:62:45:8a [ether] on qr-ae962c02-ea ? (10.0.0.9) at fa:16:3e:f1:25:e3 [ether] on qr-ae962c02-ea gateway (172.24.4.1) at 12:c0:86:a9:b0:49 [ether] on qg-9452f9cc-51
测试下
[lihui@openstack ~]$ sudo ip netns exec qrouter-70ad5580-33a1-4f77-b242-1c3d2b71af8c arping -I qg-9452f9cc-51 172.24.4.9 ARPING 172.24.4.9 from 172.24.4.2 qg-9452f9cc-51 ^CSent 20 probes (20 broadcast(s)) Received 0 response(s)
在qvo口上抓ARP，有去无回
[lihui@openstack ~]$ sudo tcpdump -i qvoa08c3748-20 arp -en tcpdump: WARNING: qvoa08c3748-20: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qvoa08c3748-20, link-type EN10MB (Ethernet), capture size 65535 bytes 01:33:55.544333 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:33:56.545358 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:33:57.548094 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:33:58.551932 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:33:59.552289 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:34:00.552910 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:34:01.553441 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:34:02.554331 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:34:03.555434 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:34:04.555929 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:34:05.556140 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28 01:34:06.556483 fa:16:3e:56:98:50 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.9 (Broadcast) tell 172.24.4.2, length 28
这就坑了，又没法VNC登进虚拟机，突然想到，namespace里不是可以连通内网IP么，直接登陆
[lihui@openstack devstack]$ sudo ip netns exec qrouter-70ad5580-33a1-4f77-b242-1c3d2b71af8c ssh -i dev.private cirros@10.0.0.9
$
$
$ ip a
1: lo: mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1450 qdisc pfifo_fast qlen 1000
link/ether fa:16:3e:f1:25:e3 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.9/26 brd 10.0.0.63 scope global eth0
inet6 fd8d:4a3:aeb0:0:f816:3eff:fef1:25e3/64 scope global dynamic
valid_lft 86389sec preferred_lft 14389sec
inet6 fe80::f816:3eff:fef1:25e3/64 scope link
valid_lft forever preferred_lft forever
3: eth1: mtu 1500 qdisc noop qlen 1000
link/ether fa:16:3e:3c:61:a9 brd ff:ff:ff:ff:ff:ff
$ arp -a
host-10-0-0-1.openstacklocal (10.0.0.1) at fa:16:3e:97:da:84 [ether] on eth0
host-10-0-0-2.openstacklocal (10.0.0.2) at fa:16:3e:62:45:8a [ether] on eth0
好事，登陆进来了，对比eth1的MAC地址和上面流表in_port=20的MAC地址，发现是一致的，也就是说这个eth1应该就是public IP地址，至于没有UP以及分配IP地址的原因应该是cirros镜像的问题，手动UP并指定IP地址
$ sudo ifconfig eth1 172.24.4.9/24 up
$ ip a show eth1
3: eth1: mtu 1500 qdisc pfifo_fast qlen 1000
link/ether fa:16:3e:3c:61:a9 brd ff:ff:ff:ff:ff:ff
inet 172.24.4.9/16 brd 172.24.255.255 scope global eth1
inet6 fe80::f816:3eff:fe3c:61a9/64 scope link
valid_lft forever preferred_lft forever
$
namespace外面，终于也能够通虚拟机了
[lihui@openstack ~]$ ping 172.24.4.9 PING 172.24.4.9 (172.24.4.9) 56(84) bytes of data. 64 bytes from 172.24.4.9: icmp_seq=1 ttl=64 time=1.08 ms 64 bytes from 172.24.4.9: icmp_seq=2 ttl=64 time=0.579 ms 64 bytes from 172.24.4.9: icmp_seq=3 ttl=64 time=1.02 ms 64 bytes from 172.24.4.9: icmp_seq=4 ttl=64 time=0.686 ms 64 bytes from 172.24.4.9: icmp_seq=5 ttl=64 time=0.936 ms 64 bytes from 172.24.4.9: icmp_seq=6 ttl=64 time=1.29 ms ^C --- 172.24.4.9 ping statistics --- 6 packets transmitted, 6 received, 0% packet loss, time 5007ms
密钥登陆
[lihui@openstack devstack]$ ssh -i dev.private cirros@172.24.4.9 The authenticity of host '172.24.4.9 (172.24.4.9)' can't be established. RSA key fingerprint is 04:ed:ac:47:1c:f0:1b:df:8f:7e:22:e1:3a:a1:e5:ca. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '172.24.4.9' (RSA) to the list of known hosts. @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions 0664 for 'dev.private' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. bad permissions: ignore key: dev.private cirros@172.24.4.9's password:
设置600，再次登陆，搞定
[lihui@openstack devstack]$ chmod 600 dev.private
[lihui@openstack devstack]$ ssh -i dev.private cirros@172.24.4.9
$ ip a
1: lo: mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1450 qdisc pfifo_fast qlen 1000
link/ether fa:16:3e:f1:25:e3 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.9/26 brd 10.0.0.63 scope global eth0
inet6 fd8d:4a3:aeb0:0:f816:3eff:fef1:25e3/64 scope global dynamic
valid_lft 86399sec preferred_lft 14399sec
inet6 fe80::f816:3eff:fef1:25e3/64 scope link
valid_lft forever preferred_lft forever
3: eth1: mtu 1500 qdisc pfifo_fast qlen 1000
link/ether fa:16:3e:3c:61:a9 brd ff:ff:ff:ff:ff:ff
inet 172.24.4.9/16 brd 172.24.255.255 scope global eth1
inet6 fe80::f816:3eff:fe3c:61a9/64 scope link
valid_lft forever preferred_lft forever
$
如此一来就可以直接登陆虚拟机了，至于VNC的问题有空再看
