Tempest框架RESTful流程:HTTP接口组装终极无脑版

Tempest里进行接口测试,有API HTTP的,也有Command Line的,这里简单看下API层面HTTP RESTful的调用流程

这里随便看一个tempest里的test case,比如还是test_flavors.py里

@attr(type='smoke')
def test_list_flavors(self):
# List of all flavors should contain the expected flavor
resp, flavors = self.client.list_flavors()
resp, flavor = self.client.get_flavor_details(self.flavor_ref)
flavor_min_detail = {'id': flavor['id'], 'links': flavor['links'],
'name': flavor['name']}
self.assertIn(flavor_min_detail, flavors)

根据函数test_list_flavors的命名就可以看出是为了让nose等可以直接触发测试,其实这个case要测试的内容比较简单

首先看下list_flavors方法

def list_flavors(self, params=None):
url = 'flavors'
if params:
url += '?%s' % urllib.urlencode(params)

resp, body = self.get(url)
body = json.loads(body)
return resp, body['flavors']

显然这里拼凑了一个RESTful API,然后调用GET方法,当然如果你对这里get方法具体调用什么库来实现的,可以继续深究

def get(self, url, headers=None):
return self.request('GET', url, headers)

看到这里就明白了,tempest里RESTful都是通过request模块来调用,好,再回到上面list_flavors方法,实际上get方法返回的是flavor list的结果,有兴趣可以nova –debug list对比着返回的json来看,注意的是这里body[‘flavors’]是一个list,每个元素包括了每个flavor的相关信息,比如id,name,vcpu,ram,disk等等,至于另外一个返回值resp,这里不用管了,这个case用不上,最后返回给resp和flavors

再继续看test_list_flavors,self.flavor_ref是从tempest.conf里读取的

cls.flavor_ref = cls.config.compute.flavor_ref

是由用户来进行设置进去的你需要测试的flavor,以及后续创建虚拟机也会用到,这里代码就不更深一步贴了,读写配置文件应该也会有封装好的现成的方法,继续看get_flavor_details

def get_flavor_details(self, flavor_id):
resp, body = self.get("flavors/%s" % str(flavor_id))
body = json.loads(body)
return resp, body['flavor']

这里同样执行了get方法,看到这里,发现了一件诡异的事,这里传入的URL就只有一个flavors/$flavor_id,回头检查一下上面的list_flavors方法,params为None,因此url += 这里根本不会执行,也就是一样get方法第一个参数并没有指明API的host,port,version等,因此就继续看下get调用的request的实现好了

def request(self, method, url,
headers=None, body=None):
retry = 0
if (self.token is None) or (self.base_url is None):
self._set_auth()

if headers is None:
headers = {}
headers['X-Auth-Token'] = self.token

resp, resp_body = self._request(method, url,
headers=headers, body=body)

while (resp.status == 413 and
'retry-after' in resp and
not self.is_absolute_limit(
resp, self._parse_resp(resp_body)) and
retry < MAX_RECURSION_DEPTH):
retry += 1
delay = int(resp['retry-after'])
time.sleep(delay)
resp, resp_body = self._request(method, url,
headers=headers, body=body)
self._error_checker(method, url, headers, body,
resp, resp_body)
return resp, resp_body

看到这里,原来request也是tempest里自己又封装了一个方法,而不是原始的request,其实从get里调用的是self.request也早就应该看出来了,但是这里继续没有结束,调用的还是封装的self._request方法,继续看

def _request(self, method, url,
headers=None, body=None):
"""A simple HTTP request interface."""

req_url = "%s/%s" % (self.base_url, url)
self._log_request(method, req_url, headers, body)
resp, resp_body = self.http_obj.request(req_url, method,
headers=headers, body=body)
self._log_response(resp, resp_body)
self.response_checker(method, url, headers, body, resp, resp_body)

return resp, resp_body

我们关注的是URL,这里才能找到正确答案,默认前面加了一段base_url,因此继续找下这个变量哪里进行的赋值

def _set_auth(self):
"""
Sets the token and base_url used in requests based on the strategy type
“""

if self.auth_version == 'v3':
auth_func = self.identity_auth_v3
else:
auth_func = self.keystone_auth

self.token, self.base_url = (
auth_func(self.user, self.password, self.auth_url,
self.service, self.tenant_name))

我们用的不是v3,因此找auth_func第二个返回值,也就是keystone_auth第二个返回值;同时关注下函数传入的第三个参数self.auth_url

来自于类的初始化函数

class RestClient(object):
TYPE = "json"
LOG = logging.getLogger(__name__)

def __init__(self, config, user, password, auth_url, tenant_name=None,
auth_version='v2'):
self.config = config
self.user = user
self.password = password
self.auth_url = auth_url

虽然大概能猜到base_url可能是又auth_url + xxx拼凑起来的,从命名可以看出来,但是,为了了解一下具体的实现,还是继续先找下auth_url的组成,不过这找起来就比较无聊了,__init__传入的数据,就直接不停地找派生出来的类传入的参数即可

class FlavorsClientJSON(RestClient):

def __init__(self, config, username, password, auth_url, tenant_name=None):
super(FlavorsClientJSON, self).__init__(config, username, password,
auth_url, tenant_name)

首先找到了FlavorClientJson,可是还不够,因为auth_url同样是__init__传入的,只好继续找哪里用了这个类

self.flavors_client = FlavorsClientJSON(*client_args)

终于,应该看下client_args就可以找到答案

class Manager(object):

"""
Top level manager for OpenStack Compute clients
"""

def __init__(self, username=None, password=None, tenant_name=None,
interface='json'):
"""
We allow overriding of the credentials used within the various
client classes managed by the Manager object. Left as None, the
standard username/password/tenant_name is used.

:param username: Override of the username
:param password: Override of the password
:param tenant_name: Override of the tenant name
"""
self.config = CONF
# If no creds are provided, we fall back on the defaults
# in the config file for the Compute API.
self.username = username or CONF.identity.username
self.password = password or CONF.identity.password
self.tenant_name = tenant_name or CONF.identity.tenant_name

if None in (self.username, self.password, self.tenant_name):
msg = ("Missing required credentials. "
"username: %(u)s, password: %(p)s, "
"tenant_name: %(t)s" %
{'u': username, 'p': password, 't': tenant_name})
raise exceptions.InvalidConfiguration(msg)

self.auth_url = CONF.identity.uri
self.auth_url_v3 = CONF.identity.uri_v3

client_args = (CONF, self.username, self.password,
self.auth_url, self.tenant_name)

看到这里就明白了,auth_url是从tempest.conf里配置的uri字段获取的,简单看了下配置文件,不出意料,果然是auth的url

uri = http://xx.xx.xx.xx:5000/v2.0/

先不说话,看接下来怎么发展,keystone_auth如何将auth_url变成base_url

def keystone_auth(self, user, password, auth_url, service, tenant_name):
"""
Provides authentication via Keystone using v2 identity API.
"""

# Normalize URI to ensure /tokens is in it.
if 'tokens' not in auth_url:
auth_url = auth_url.rstrip('/') + '/tokens'

creds = {
'auth': {
'passwordCredentials': {
'username': user,
'password': password,
},
'tenantName': tenant_name,
}
}

headers = {'Content-Type': 'application/json'}
body = json.dumps(creds)
self._log_request('POST', auth_url, headers, body)
resp, resp_body = self.http_obj.request(auth_url, 'POST',
headers=headers, body=body)
self._log_response(resp, resp_body)

if resp.status == 200:
try:
auth_data = json.loads(resp_body)['access']
token = auth_data['token']['id']
except Exception as e:
print("Failed to obtain token for user: %s" % e)
raise

mgmt_url = None
for ep in auth_data['serviceCatalog']:
if ep["type"] == service:
for _ep in ep['endpoints']:
if service in self.region and \
_ep['region'] == self.region[service]:
mgmt_url = _ep[self.endpoint_url]
if not mgmt_url:
mgmt_url = ep['endpoints'][0][self.endpoint_url]

if service == self.config.images.catalog_type:
if mgmt_url.endswith(r'/v1'):
mgmt_url = mgmt_url.rstrip(r'/v1')
break
(..........省略.............)

if mgmt_url is None:
raise exceptions.EndpointNotFound(service)

return token, mgmt_url

elif resp.status == 401:
raise exceptions.AuthenticationFailure(user=user,
password="******",
tenant=tenant_name)
raise exceptions.IdentityError('Unexpected status code {0}'.format(
resp.status))

这段代码建议不要凭空来看,结合keystone token-get和endpoint-list来对比结果,甚至可以这样

$ curl -s -X POST http://xx.xx.xx.xx:5000/v2.0/tokens -H "Content-Type: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "Project_bianhuabai@163.com", "passwordCredentials": {"username": "bianhuabai@163.com", "password": "xxxxxx"}}}' | jq .access.serviceCatalog | grep type
"type": "compute",
"type": "network",
"type": "image",
"type": "metering",
"type": "volume",
"type": "dns",
"type": "identity",

总体流程就是,先获取token,这里根据不同的服务,返回的结果就是publicURL的value

self.endpoint_url = 'publicURL'

也就是这些

$ keystone endpoint-list | awk '{print $6}'

publicurl

http://xx.xx.xx.xx:5000/v2.0
http://xx.xx.xx.xx:5000/v2.0
http://xx.xx.xx.xx:8777/
http://xx.xx.xx.xx:9001/
http://xx.xx.xx.xx:8774/v2/$(tenant_id)s
http://xx.xx.xx.xx:9292/v1
http://xx.xx.xx.xx:9696
http://xx.xx.xx.xx:9292/v1
http://xx.xx.xx.xx:8776/v1/$(tenant_id)s
http://xx.xx.xx.xx:8774/v2/$(tenant_id)s
http://xx.xx.xx.xx:9696
http://xx.xx.xx.xx:8776/v1/$(tenant_id)s

但何种服务如何判断呢?这个问题比较重要,keystone_auth里传入了一个service参数,还是这里

self.token, self.base_url = (
auth_func(self.user, self.password, self.auth_url,
self.service, self.tenant_name))

而self.service赋值的地方,在每个对应服务封装好的接口中,具体可查看tempest/service目录下,由于对应服务已经确定了,因此这里self.service的结果也是确定的,比如flavor这个接口

class FlavorsClientJSON(RestClient):

def __init__(self, config, username, password, auth_url, tenant_name=None):
super(FlavorsClientJSON, self).__init__(config, username, password,
auth_url, tenant_name)
self.service = self.config.compute.catalog_type

最终能够找到从配置里面读取compute

self.compute = cfg.CONF.compute

然后再回到_request里

req_url = "%s/%s" % (self.base_url, url)

self.base_url就是上面将compute作为参数传入keystone_auth里,得到compute服务的base_url,url是get方法调用的地方,所以那时候才只需要传一个flavor,而不需要带前面的host和port,这就是整个RESTful的流程

 

再次回到现实当中,这个case就十分简单了

def test_list_flavors(self):
# List of all flavors should contain the expected flavor
resp, flavors = self.client.list_flavors()
resp, flavor = self.client.get_flavor_details(self.flavor_ref)
flavor_min_detail = {'id': flavor['id'], 'links': flavor['links'],
'name': flavor['name']}
self.assertIn(flavor_min_detail, flavors)

首先列出所有flavor信息,这里返回的是列表flavors,然后获取我们配置文件里配置好的flavor_ref的相关信息给flavor,但这里保留了三个flavor最少的信息,id,links和name,最终验证的就是这三个信息都在全集flavors里,至于说这里封装的self.assertIn到底是key/value都要进行检查还是怎么样的,就不看了 

发表评论