吐血部署社区Liberty版本devstack单节点openstack

by

2017-10-06

17:02:32

Leave a comment on 吐血部署社区Liberty版本devstack单节点openstack

OpenStack

, ,

本来想自己搭一个openstack玩玩vpnaas，可devstack里vpnaas已经在M版本被移除了，具体如下

commit 1d4303db4e1f194dd4c2284f6dd4189f41f8d2f3
Merge: 3e5b495 95cb2ea
Author: Jenkins
Date:   Wed May 11 03:16:22 2016 +0000

    Merge "Remove vpnaas code from devstack" into stable/mitaka

commit 95cb2ea68f7e12a3e7baf627a2c37e77caeea294
Author: Ihar Hrachyshka
Date:   Thu Apr 28 12:39:26 2016 -0700

    Remove vpnaas code from devstack

    This service is now configured by devstack plugin in master.

    Change-Id: Ie5fc0d2a45c1b564f98c69ec9ea6fbdeeb465d32
    (cherry picked from commit 1a791cbc449a3cfdbc3d3b94c6cda3ddefa17af4)

用最新的P版本折腾了会，vpnaas作为plugin还是没搞明白如何搭建，于是直接搭一个Liberty版本，带VPNaaS的玩耍下，没想到这么折腾，记录一下心酸旅程

1：环境

我这里用ubuntu14.04都部署成功了，centos6.6中途卡主，暂未解决，debian7.9安装部署完最后ovs-agent不明原因起不来；添加一个普通用户，添加sudo权限，设置sudo不需要密码

2：devstack下载

git clone https://github.com/openstack-dev/devstack.git -b liberty-eol

3：修改源，分支

这里改成douban源，但是最好/root/.pip/pip.conf和/home/$user/.pip/pip.conf都添加配置

[global]
trusted-host=pypi.douban.com
index-url = http://pypi.douban.com/simple

devstack下载的是L版，但是stack.sh里会重新从github上下nova，cinder等模块，默认是从发最新master上下！！！！！因此需要修改相关配置，具体可github上查看相关分支，也都是在tag里，操作如下

sed -i "s/stable\/liberty/liberty-eol/g" lib/ironic
sed -i "s/stable\/liberty/liberty-eol/g" stackrc
sed -i "s/stable\/liberty/liberty-eol/g" tests/test_refs.sh

将stable/liberty都替换成liberty-eol

lihui@l-openstack:~/devstack$ grep liberty-eol ./* -R
./lib/ironic:IPA_DOWNLOAD_BRANCH=${IPA_DOWNLOAD_BRANCH:-liberty-eol}
./stackrc:CINDER_BRANCH=${CINDER_BRANCH:-liberty-eol}
./stackrc:GLANCE_BRANCH=${GLANCE_BRANCH:-liberty-eol}
./stackrc:HEAT_BRANCH=${HEAT_BRANCH:-liberty-eol}
./stackrc:HORIZON_BRANCH=${HORIZON_BRANCH:-liberty-eol}
./stackrc:IRONIC_BRANCH=${IRONIC_BRANCH:-liberty-eol}
./stackrc:KEYSTONE_BRANCH=${KEYSTONE_BRANCH:-liberty-eol}
./stackrc:NEUTRON_BRANCH=${NEUTRON_BRANCH:-liberty-eol}
./stackrc:NEUTRON_FWAAS_BRANCH=${NEUTRON_FWAAS_BRANCH:-liberty-eol}
./stackrc:NEUTRON_LBAAS_BRANCH=${NEUTRON_LBAAS_BRANCH:-liberty-eol}
./stackrc:NEUTRON_VPNAAS_BRANCH=${NEUTRON_VPNAAS_BRANCH:-liberty-eol}
./stackrc:NOVA_BRANCH=${NOVA_BRANCH:-liberty-eol}
./stackrc:SWIFT_BRANCH=${SWIFT_BRANCH:-liberty-eol}
./stackrc:REQUIREMENTS_BRANCH=${REQUIREMENTS_BRANCH:-liberty-eol}
./tests/test_refs.sh:REFS=`grep BRANCH stackrc | grep -v -- '-master' | grep -v -- '-liberty-eol'`

4：注释掉tempest相关

tempest因为并没有老的分支，版本较新，安装过程中问题较多，注释掉；修改stackrc里

# This allows us to pass ``ENABLED_SERVICES``
if ! isset ENABLED_SERVICES ; then
    # Keystone - nothing works without keystone
    ENABLED_SERVICES=key
    # Nova - services to support libvirt based openstack clouds
    ENABLED_SERVICES+=,n-api,n-cpu,n-net,n-cond,n-sch,n-novnc,n-crt,n-cauth
    # Glance services needed for Nova
    ENABLED_SERVICES+=,g-api,g-reg
    # Cinder
    ENABLED_SERVICES+=,c-sch,c-api,c-vol
    # Dashboard
    ENABLED_SERVICES+=,horizon
    # Additional services
    ENABLED_SERVICES+=,rabbit,mysql,dstat
    #ENABLED_SERVICES+=,rabbit,tempest,mysql,dstat
fi

5：修改集中式为分部署dvr，lib/neutron-legacy里修改为dvr_snat

# Distributed Virtual Router (DVR) configuration
# Can be:
# - ``legacy``   - No DVR functionality
# - ``dvr_snat`` - Controller or single node DVR
# - ``dvr``      - Compute node in multi-node DVR
#
Q_DVR_MODE=${Q_DVR_MODE:-dvr_snat}
if [[ "$Q_DVR_MODE" != "legacy" ]]; then
    Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch,linuxbridge,l2population
fi

ubuntu的，经过这几步修改，只要网络不太烂，应该就可以搭建成功

6：debian操作系统有一个问题，关于tgt

在lib/cinder里start_cinder函数里，有一段：

if [ "$CINDER_ISCSI_HELPER" = "tgtadm" ]; then
        if is_service_enabled c-vol; then
            # Delete any old stack.conf
            sudo rm -f /etc/tgt/conf.d/stack.conf
            _configure_tgt_for_config_d
            if is_ubuntu; then
                sudo service tgt restart
            elif is_suse; then
                # NOTE(dmllr): workaround restart bug
                # https://bugzilla.suse.com/show_bug.cgi?id=934642
                stop_service tgtd
                start_service tgtd
            else
                restart_service tgtd
            fi
            # NOTE(gfidente): ensure tgtd is running in debug mode
            sudo tgtadm --mode system --op update --name debug --value on
        fi
    fi

这里is_ubuntu为真，但是service执行tgt的时候报错；原因是debian操作系统并没有生成一个tgt命名的开机启动脚本，具体来说就是/etc/init.d里没有tgt，而ubuntu里是有的；但是具体tgtd可执行程序是有的（真奇葩的命名！tgtd是可执行程序，tgt是开机服务）这里有两种方法解决：

（1）修改lib/cinder里所有start，stop，restart tgtd的命令，直接执行可执行程序，而不是开机启动服务，但是需要修改的地方较多

（2）直接添加一个开机服务脚本tgt，一劳永逸，也可以直接从ubuntu copy，如下

#!/bin/bash
### BEGIN INIT INFO
# Provides:          tgtd
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Should-Start:      zfs
# Should-Stop:       zfs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: iscsi target daemon
# Description:       iscsi target daemon
### END INIT INFO

DESC="target framework daemon"
NAME=tgtd
DAEMON=/usr/sbin/${NAME}

TGTD_CONFIG=/etc/tgt/targets.conf

TASK=$1

. /lib/lsb/init-functions

[ -x $DAEMON ] || exit 0

start()
{
	log_daemon_msg "Starting $DESC" "$NAME"
	# Start tgtd first.
	tgtd &>/dev/null
	RETVAL=$?
	if [ "$RETVAL" -ne 0 ] ; then
		log_end_msg 1
		exit 1
	else
		log_end_msg 0
	fi
	# Put tgtd into "offline" state until all the targets are configured.
	# We don't want initiators to (re)connect and fail the connection
	# if it's not ready.
	tgtadm --op update --mode sys --name State -v offline
	# Configure the targets.
	tgt-admin -e -c $TGTD_CONFIG
	# Put tgtd into "ready" state.
	tgtadm --op update --mode sys --name State -v ready
}

stop()
{
	if [ "$RUNLEVEL" == 0 -o "$RUNLEVEL" == 6 ] ; then
	    forcedstop
	fi
	log_daemon_msg "Stopping $DESC" "$NAME"
	# Remove all targets. It only removes targets which are not in use.
	tgt-admin --update ALL -c /dev/null &>/dev/null
	# tgtd will exit if all targets were removed
	tgtadm --op delete --mode system &>/dev/null
	RETVAL=$?
	if [ "$RETVAL" -eq 107 ] ; then
		if [ "$TASK" != "restart" ] ; then
			log_end_msg 1
			exit 1
		else
			log_end_msg 0
		fi
	elif [ "$RETVAL" -ne 0 ] ; then
		log_end_msg 1
		echo "Some initiators are still connected - could not stop tgtd"
		exit 2
	else
		log_end_msg 0
	fi
	echo -n
}

forcedstop()
{
	# NOTE: Forced shutdown of the iscsi target may cause data corruption
	# for initiators that are connected.
	echo "Force-stopping target framework daemon"
	# Offline everything first. May be needed if we're rebooting, but
	# expect the initiators to reconnect cleanly when we boot again
	# (i.e. we don't want them to reconnect to a tgtd which is still
	# working, but the target is gone).
	tgtadm --op update --mode sys --name State -v offline &>/dev/null
	RETVAL=$?
	if [ "$RETVAL" -eq 107 ] ; then
	    echo "tgtd is not running"
	    [ "$TASK" != "restart" ] && exit 1
	else
	    tgt-admin --offline ALL
	    # Remove all targets, even if they are still in use.
	    tgt-admin --update ALL -c /dev/null -f
	    # It will shut down tgtd only after all targets were removed.
	    tgtadm --op delete --mode system
	    RETVAL=$?
	    if [ "$RETVAL" -ne 0 ] ; then
		echo "Failed to shutdown tgtd"
		exit 1
	    fi
	fi
	echo -n
}

reload()
{
	log_daemon_msg "Reloading configuration of $DESC" "$NAME"
	# Update configuration for targets. Only targets which
	# are not in use will be updated.
	tgt-admin --update ALL -c $TGTD_CONFIG &>/dev/null
	RETVAL=$?
	if [ "$RETVAL" -eq 107 ] ; then
		log_end_msg 1
		echo "tgtd is not running"
		exit 1
	fi
	log_end_msg 0
}

forcedreload()
{
	log_daemon_msg "Forced-reload configuration of $DESC" "$NAME"
	# Update configuration for targets, even those in use.
	tgt-admin --update ALL -f -c $TGTD_CONFIG &>/dev/null
	RETVAL=$?
	if [ "$RETVAL" -eq 107 ] ; then
		log_end_msg 1
		echo "tgtd is not running"
		exit 1
	else
		log_end_msg 0
	fi
}

status()
{
	# Don't name this script "tgtd"...
	TGTD_PROC=$(ps -C tgtd | grep -c tgtd)
	if [ "$TGTD_PROC" -eq 2 ] ; then
	    echo "tgtd is running. Run 'tgt-admin -s' to see detailed target info."
	else
	    echo "tgtd is NOT running."
	fi
}

case $1 in
	start)
		start
		;;
	stop)
		stop
		;;
	forcedstop)
		forcedstop
		;;
	restart)
		TASK=restart
		stop && start
		;;
	forcedrestart)
		TASK=restart
		forcedstop && start
		;;
	reload)
		reload
		;;
	force-reload)
		forcedreload
		;;
	status)
		status
		;;
	*)
		echo "Usage: $0 {start|stop|forcedstop|restart|forcedrestart|reload|force-reload|status}"
		exit 2
		;;
esac

到了这里，我以为大功告成了，测试过程中又发现两问题，一个是neutron居然都没有安装，另一个是所有的client全部都是装的master，不知道会不会有不匹配的，一个一个来解决

7：neutron安装

 首先install_neutron在lib/neutron-legacy里，然后必须要先enable neutron service才行

if is_service_enabled neutron; then
    install_neutron_agent_packages
fi

查看stackrc配置，可以看到如下说明

# In order to enable Neutron (a single node setup) add the following
# settings in ``local.conf``:
#  [[local|localrc]]
#  disable_service n-net
#  enable_service q-svc
#  enable_service q-agt
#  enable_service q-dhcp
#  enable_service q-l3
#  enable_service q-meta
#  # Optional, to enable tempest configuration as part of DevStack
#  enable_service tempest

按要求添加到local.conf里，然后看下is_service_enabled函数，实现如下

# Uses global ``ENABLED_SERVICES``
# is_service_enabled service [service ...]
function is_service_enabled {
    local xtrace=$(set +o | grep xtrace)
    set +o xtrace
    local enabled=1
    local services=$@
    local service
    for service in ${services}; do
        [[ ,${ENABLED_SERVICES}, =~ ,${service}, ]] && enabled=0

        # Look for top-level 'enabled' function for this service
        if type is_${service}_enabled >/dev/null 2>&1; then
            # A function exists for this service, use it
            is_${service}_enabled && enabled=0
        fi

        # TODO(dtroyer): Remove these legacy special-cases after the is_XXX_enabled()
        #                are implemented

        [[ ${service} == n-cell-* && ${ENABLED_SERVICES} =~ "n-cell" ]] && enabled=0
        [[ ${service} == n-cpu-* && ${ENABLED_SERVICES} =~ "n-cpu" ]] && enabled=0
        [[ ${service} == "nova" && ${ENABLED_SERVICES} =~ "n-" ]] && enabled=0
        [[ ${service} == "glance" && ${ENABLED_SERVICES} =~ "g-" ]] && enabled=0
        [[ ${service} == "ironic" && ${ENABLED_SERVICES} =~ "ir-" ]] && enabled=0
        [[ ${service} == "neutron" && ${ENABLED_SERVICES} =~ "q-" ]] && enabled=0
        [[ ${service} == "trove" && ${ENABLED_SERVICES} =~ "tr-" ]] && enabled=0
        [[ ${service} == "swift" && ${ENABLED_SERVICES} =~ "s-" ]] && enabled=0
        [[ ${service} == s-* && ${ENABLED_SERVICES} =~ "swift" ]] && enabled=0
    done
    $xtrace
    return $enabled
}

根据注释可以知道ENABLED_SERVICES是一个全局变量，从其它地方赋值

继续在stackrc里，可以找到初始化的地方，也是通过配置来实现，添加neutron的服务

# This allows us to pass ``ENABLED_SERVICES``
if ! isset ENABLED_SERVICES ; then
    # Keystone - nothing works without keystone
    ENABLED_SERVICES=key
    # Nova - services to support libvirt based openstack clouds
    ENABLED_SERVICES+=,n-api,n-cpu,n-net,n-cond,n-sch,n-novnc,n-crt,n-cauth
    # Glance services needed for Nova
    ENABLED_SERVICES+=,g-api,g-reg
    # Cinder
    ENABLED_SERVICES+=,c-sch,c-api,c-vol
    # Neutron
    ENABLED_SERVICES+=,q-svc,q-agt,q-dhcp,q-l3,q-meta
    # Dashboard
    ENABLED_SERVICES+=,horizon
    # Additional services
    ENABLED_SERVICES+=,rabbit,mysql,dstat
    #ENABLED_SERVICES+=,rabbit,tempest,mysql,dstat
fi

修改了这些地方之后，再执行以下stack.sh看看，终于看到了下面的流程，基本放心了

+ git_clone git://git.openstack.org/openstack/neutron.git /opt/stack/neutron liberty-eol
+ local git_remote=git://git.openstack.org/openstack/neutron.git
+ local git_dest=/opt/stack/neutron
+ local git_ref=liberty-eol
++ pwd
+ local orig_dir=/home/lihui/devstack
+ local git_clone_flags=
++ trueorfalse False RECLONE
+++ set +o
+++ grep xtrace
++ local 'xtrace=set -o xtrace'
++ set +o xtrace
+ RECLONE=False
+ [[ 0 -gt 0 ]]
+ [[ False = \T\r\u\e ]]
+ echo liberty-eol
+ egrep -q '^refs'
+ [[ ! -d /opt/stack/neutron ]]
+ [[ False = \T\r\u\e ]]
+ git_timed clone git://git.openstack.org/openstack/neutron.git /opt/stack/neutron
+ local count=0
+ local timeout=0
+ [[ -n 0 ]]
+ timeout=0
+ timeout -s SIGINT 0 git clone git://git.openstack.org/openstack/neutron.git /opt/stack/neutron
Cloning into '/opt/stack/neutron'...
remote: Counting objects: 241428, done.
remote: Compressing objects: 100% (114676/114676), done.
remote: Total 241428 (delta 177278), reused 167196 (delta 106821)
Receiving objects: 100% (241428/241428), 62.18 MiB | 9.49 MiB/s, done.
Resolving deltas: 100% (177278/177278), done.

又报了一个错误

+ neutron router-interface-add 3e1a35b2-74e3-481b-9348-4882a5a50789 8f222bed-f434-46ca-9cb8-22648b0f31de
Added interface 3e1ce181-89e9-4567-833f-3f4ac2837673 to router 3e1a35b2-74e3-481b-9348-4882a5a50789.
++ _neutron_create_public_subnet_v4 1519aded-bc3f-40f6-83ad-d88dacc064cd
++ local 'subnet_params+=--ip_version 4 '
++ subnet_params+=' '
++ subnet_params+='--gateway 172.24.4.1 '
++ subnet_params+='--name public-subnet '
++ subnet_params+='1519aded-bc3f-40f6-83ad-d88dacc064cd 7.1.1.0/24 '
++ subnet_params+='-- --enable_dhcp=False'
+++ neutron subnet-create --ip_version 4 --gateway 172.24.4.1 --name public-subnet 1519aded-bc3f-40f6-83ad-d88dacc064cd 7.1.1.0/24 -- --enable_dhcp=False
+++ grep -e gateway_ip -e ' id '
Invalid input for operation: Gateway is not valid on subnet.
++ local id_and_ext_gw_ip=
++ die_if_not_set 1280 id_and_ext_gw_ip 'Failure creating public IPv4 subnet'
++ local exitcode=0
+++ set +o
+++ grep xtrace
++ local 'xtrace=set -o xtrace'
++ set +o xtrace
[ERROR] /home/lihui/devstack/functions-common:1280 Failure creating public IPv4 subnet

手动执行以下该subnet的创建，的确有错误

lihui@l-openstack:~/devstack$  neutron subnet-create --ip_version 4 --gateway 172.24.4.1 --name public-subnet 1519aded-bc3f-40f6-83ad-d88dacc064cd 7.1.1.0/24 -- --enable_dhcp=False
Invalid input for operation: Gateway is not valid on subnet.

其中7.1.1.0/24是我自己配置的一段外网，但是外网网关却冒出来默认的172.24.4.1，可见还有其它地方配死了这个网关，全局搜索一下，的确还有一个漏网之鱼

lihui@l-openstack:~/devstack$ grep "172.24.4.1" ./* -R
./lib/neutron-legacy:PUBLIC_NETWORK_GATEWAY=${PUBLIC_NETWORK_GATEWAY:-172.24.4.1}
./tools/xen/xenrc:PUB_IP=${PUB_IP:-172.24.4.10}
lihui@l-openstack:~/devstack$
lihui@l-openstack:~/devstack$
lihui@l-openstack:~/devstack$ grep "172.24" ./* -R
./lib/neutron-legacy:PUBLIC_NETWORK_GATEWAY=${PUBLIC_NETWORK_GATEWAY:-172.24.4.1}
./tools/xen/xenrc:PUB_IP=${PUB_IP:-172.24.4.10}

将外网网关改掉，再执行一次stack.sh，又来一个错误

+ neutron_plugin_configure_plugin_agent
+ _neutron_ovs_base_setup_bridge br-int
+ local bridge=br-int
+ neutron-ovs-cleanup
2017-10-06 18:15:33.944 12993 INFO neutron.common.config [-] Logging enabled!
2017-10-06 18:15:33.945 12993 INFO neutron.common.config [-] /usr/local/bin/neutron-ovs-cleanup version 7.2.0
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--if-exists', '--columns=name,external_ids,ofport', 'list', 'Interface', 'qr-3e1ce181-89', 'sg-d69ae8ed-7a', 'tap464e2610-09'].
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl   File "/opt/stack/neutron/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl   File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 159, in execute
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError:
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['sudo', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--if-exists', '--columns=name,external_ids,ofport', 'list', 'Interface', 'qr-3e1ce181-89', 'sg-d69ae8ed-7a', 'tap464e2610-09']
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 1
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl
2017-10-06 18:15:33.989 12993 ERROR neutron.agent.ovsdb.impl_vsctl
2017-10-06 18:15:34.008 12993 CRITICAL neutron [-] RuntimeError:
Command: ['sudo', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--if-exists', '--columns=name,external_ids,ofport', 'list', 'Interface', 'qr-3e1ce181-89', 'sg-d69ae8ed-7a', 'tap464e2610-09']
Exit code: 1

看样子是在执行neutron-ovs-cleanup的时候报错；感觉不像是安装部署的问题，怀疑可能是上次出错残留，没有清理，执行以下unstack.sh，kill掉所有相关进行，再执行一次，果然就没这错误了，果然还是得靠经验，捂脸图

接着错误又来了

+ [[ -x /home/lihui/devstack/local.sh ]]
+ service_check
+ local service
+ local failures
+ SCREEN_NAME=stack
+ SERVICE_DIR=/opt/stack/status
+ [[ ! -d /opt/stack/status/stack ]]
++ ls /opt/stack/status/stack/q-agt.failure
+ failures=/opt/stack/status/stack/q-agt.failure
+ for service in '$failures'
++ basename /opt/stack/status/stack/q-agt.failure
+ service=q-agt.failure
+ service=q-agt
+ echo 'Error: Service q-agt is not running'
Error: Service q-agt is not running
+ '[' -n /opt/stack/status/stack/q-agt.failure ']'
+ die 1571 'More details about the above errors can be found with screen, with ./rejoin-stack.sh'
+ local exitcode=0
+ set +o xtrace
[Call Trace]
./stack.sh:1315:service_check
/home/lihui/devstack/functions-common:1571:die
[ERROR] /home/lihui/devstack/functions-common:1571 More details about the above errors can be found with screen, with ./rejoin-stack.sh
Error on exit
World dumping... see /opt/stack/logs/worlddump-2017-10-06-103940.txt for details

看样子是所有service的进程文件都存在了/opt/stack/status/stack下，但是q-agt生成了一个错误文件，看一下q-agt.failure的内容，只有一句话：q-agt failed to start

-rw-r--r-- 1 lihui lihui 22 Oct  6 18:37 q-agt.failure
-rw-r--r-- 1 lihui lihui  6 Oct  6 18:37 q-agt.pid
-rw-r--r-- 1 lihui lihui  6 Oct  6 18:37 q-dhcp.pid
-rw-r--r-- 1 lihui lihui  6 Oct  6 18:37 q-l3.pid
-rw-r--r-- 1 lihui lihui  6 Oct  6 18:37 q-meta.pid
-rw-r--r-- 1 lihui lihui  6 Oct  6 18:37 q-svc.pid

通过查看neutron相关服务的pid文件，q-svc对应neutron-server，q-meta对应metadata，q-l3对应l3-agent，q-dhcp对应dhcp-agent，那么q-apt肯定对应着ovs-agent

需要知道如何启动的，只有看源码了，在stack.sh里

if is_service_enabled neutron; then
    start_neutron_agents
fi

在lib/neutron-legacy里

# Start running processes, including screen
function start_neutron_agents {
    # Start up the neutron agents if enabled
    start_neutron_l2_agent
    start_neutron_other_agents
}

继续看ovs-agent

# Control of the l2 agent is separated out to make it easier to test partial
# upgrades (everything upgraded except the L2 agent)
function start_neutron_l2_agent {
    run_process q-agt "$AGENT_BINARY --config-file $NEUTRON_CONF --config-file /$Q_PLUGIN_CONF_FILE"

    if is_provider_network; then
        sudo ovs-vsctl --no-wait -- --may-exist add-port $OVS_PHYSICAL_BRIDGE $PUBLIC_INTERFACE
        sudo ip link set $OVS_PHYSICAL_BRIDGE up
        sudo ip link set br-int up
        sudo ip link set $PUBLIC_INTERFACE up
        if is_ironic_hardware; then
            for IP in $(ip addr show dev $PUBLIC_INTERFACE | grep ' inet ' | awk '{print $2}'); do
                sudo ip addr del $IP dev $PUBLIC_INTERFACE
                sudo ip addr add $IP dev $OVS_PHYSICAL_BRIDGE
            done
            sudo ip route replace $FIXED_RANGE via $NETWORK_GATEWAY dev $OVS_PHYSICAL_BRIDGE
        fi
    fi
}

按照这里run_process手动起一下ovs-agent

lihui@l-openstack:~/devstack$ sudo /usr/local/bin/neutron-openvswitch-agent --config-file /etc/neutron/neutron.conf --config-file=/etc/neutron/plugins/ml2/ml2_conf.ini

还是报错

Exit code: 1
 from (pid=1956) execute /opt/stack/neutron/neutron/agent/linux/utils.py:156
2017-10-06 19:06:09.367 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set', 'Bridge', 'br-int', 'protocols=[OpenFlow10]'].
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl   File "/opt/stack/neutron/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl   File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 159, in execute
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl RuntimeError:
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl Command: ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set', 'Bridge', 'br-int', 'protocols=[OpenFlow10]']
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl Exit code: 1
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl
2017-10-06 19:06:09.367 TRACE neutron.agent.ovsdb.impl_vsctl
2017-10-06 19:06:09.368 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-]
Command: ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set', 'Bridge', 'br-int', 'protocols=[OpenFlow10]']
Exit code: 1
 Agent terminated!
2017-10-06 19:06:09.370 INFO oslo_rootwrap.client [-] Stopping rootwrap daemon process with pid=2064

debian上这个错误看了半天，日志里也是这个错误，不明觉厉；但是ubuntu14.04上无误，不清楚是否还是debian上一次调试错误，还是有ovs port残留导致？？

 

总之，最终ubuntu上看到这种信息还是令人愉快的

This is your host IP address: 10.166.224.212
This is your host IPv6 address: ::1
Horizon is now available at http://10.166.224.212/dashboard
Keystone is serving at http://10.166.224.212:5000/
The default users are: admin and demo
The password: lihui
2017-10-06 11:19:27.276 | stack.sh completed in 584 seconds.

这还不够，所有neutron的agent都起来了

lihui@l-openstack:~/devstack$ neutron agent-list
+--------------------------------------+--------------------+-------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host        | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+-------------+-------+----------------+---------------------------+
| 01bce5b5-88f5-4445-b85b-f585ec518f13 | L3 agent           | l-openstack | :-)   | True           | neutron-l3-agent          |
| 14c284ac-4e90-4329-ad18-0e364da2d71d | DHCP agent         | l-openstack | :-)   | True           | neutron-dhcp-agent        |
| 52ef51f8-d7df-463e-9de7-342f4de6af2b | Metadata agent     | l-openstack | :-)   | True           | neutron-metadata-agent    |
| fbfad2d9-2ae8-444d-83f8-2470933c6f75 | Open vSwitch agent | l-openstack | :-)   | True           | neutron-openvswitch-agent |
+--------------------------------------+--------------------+-------------+-------+----------------+---------------------------+

但是VPC已经是DVR了，创建vpn-service，却还让我带subnet，不明觉厉

lihui@l-openstack:~/devstack$ neutron vpn-service-create 060d1a6f-47fa-4d38-a617-15c04b6f59ff
usage: neutron vpn-service-create [-h]
                                  [-f {html,json,json,shell,table,value,yaml,yaml}]
                                  [-c COLUMN] [--max-width ]
                                  [--noindent] [--prefix PREFIX]
                                  [--request-format {json,xml}]
                                  [--tenant-id TENANT_ID] [--admin-state-down]
                                  [--name NAME] [--description DESCRIPTION]
                                  ROUTER SUBNET
neutron vpn-service-create: error: too few arguments

难不成neutron client只支持集中式？到哪里vpn都是坑啊~！ 

8：CLI分支

把各个模块client也换成libverty-eol

##############
#
#  OpenStack Client Library Components
#
##############

# volume client
GITREPO["python-cinderclient"]=${CINDERCLIENT_REPO:-${GIT_BASE}/openstack/python-cinderclient.git}
GITBRANCH["python-cinderclient"]=${CINDERCLIENT_BRANCH:-liberty-eol}

# python glance client library
GITREPO["python-glanceclient"]=${GLANCECLIENT_REPO:-${GIT_BASE}/openstack/python-glanceclient.git}
GITBRANCH["python-glanceclient"]=${GLANCECLIENT_BRANCH:-liberty-eol}

# python heat client library
GITREPO["python-heatclient"]=${HEATCLIENT_REPO:-${GIT_BASE}/openstack/python-heatclient.git}
GITBRANCH["python-heatclient"]=${HEATCLIENT_BRANCH:-liberty-eol}

# ironic client
GITREPO["python-ironicclient"]=${IRONICCLIENT_REPO:-${GIT_BASE}/openstack/python-ironicclient.git}
GITBRANCH["python-ironicclient"]=${IRONICCLIENT_BRANCH:-liberty-eol}

# the base authentication plugins that clients use to authenticate
GITREPO["keystoneauth"]=${KEYSTONEAUTH_REPO:-${GIT_BASE}/openstack/keystoneauth.git}
GITBRANCH["keystoneauth"]=${KEYSTONEAUTH_BRANCH:-liberty-eol}

# python keystone client library to nova that horizon uses
GITREPO["python-keystoneclient"]=${KEYSTONECLIENT_REPO:-${GIT_BASE}/openstack/python-keystoneclient.git}
GITBRANCH["python-keystoneclient"]=${KEYSTONECLIENT_BRANCH:-liberty-eol}

# neutron client
GITREPO["python-neutronclient"]=${NEUTRONCLIENT_REPO:-${GIT_BASE}/openstack/python-neutronclient.git}
GITBRANCH["python-neutronclient"]=${NEUTRONCLIENT_BRANCH:-liberty-eol}

# python client library to nova that horizon (and others) use
GITREPO["python-novaclient"]=${NOVACLIENT_REPO:-${GIT_BASE}/openstack/python-novaclient.git}
GITBRANCH["python-novaclient"]=${NOVACLIENT_BRANCH:-liberty-eol}

# python swift client library
GITREPO["python-swiftclient"]=${SWIFTCLIENT_REPO:-${GIT_BASE}/openstack/python-swiftclient.git}
GITBRANCH["python-swiftclient"]=${SWIFTCLIENT_BRANCH:-liberty-eol}

# consolidated openstack python client
GITREPO["python-openstackclient"]=${OPENSTACKCLIENT_REPO:-${GIT_BASE}/openstack/python-openstackclient.git}
GITBRANCH["python-openstackclient"]=${OPENSTACKCLIENT_BRANCH:-liberty-eol}
# this doesn't exist in a lib file, so set it here
GITDIR["python-openstackclient"]=$DEST/python-openstackclient

这样，基本就全部搞定了

发表回复