Unittest里反向测试以及又一个莫名其妙的Case

首先是瞄到了一个用例

    @test.attr(type=['negative', 'gate'])
    def test_show_host_detail_with_non_admin_user(self):
        hostname = self._get_host_name()

        self.assertRaises(exceptions.Unauthorized,
                          self.non_admin_client.show_host_detail,
                          hostname)

最后一行assert格外引人注意,因为这里参数传了仨,而且第一个参数看上去是未许可权限不够的节奏,看这意思应该是后面两个参数无论干了啥,结果会触发异常认证失败,看下这个方法的定义,在testtools.testcase.py:TestCase里

    def assertRaises(self, excClass, callableObj, *args, **kwargs):
        """Fail unless an exception of class excClass is thrown
           by callableObj when invoked with arguments args and keyword
           arguments kwargs. If a different type of exception is
           thrown, it will not be caught, and the test case will be
           deemed to have suffered an error, exactly as for an
           unexpected exception.
        """
        class ReRaiseOtherTypes(object):
            def match(self, matchee):
                if not issubclass(matchee[0], excClass):
                    reraise(*matchee)
        class CaptureMatchee(object):
            def match(self, matchee):
                self.matchee = matchee[1]
        capture = CaptureMatchee()
        matcher = Raises(MatchesAll(ReRaiseOtherTypes(),
                MatchesException(excClass), capture))
        our_callable = Nullary(callableObj, *args, **kwargs)
        self.assertThat(our_callable, matcher)
        return capture.matchee
    failUnlessRaises = assertRaises

从注释可以看出来是验证异常错误信息的正确性,参数分别是预期的异常信息,后面参数就是具体要执行的操作,或者是函数方法,含义就是验证self.non_admin_client.show_host_detail(hostname)是否是触发exceptions.Unauthorized这个异常信息

也就是这个用例本来执行就不会成功,但是会有返回异常,从而验证预期的异常信息与实际捕获的异常进行对比验证

看一个更简单的例子,也是网上比较喜欢用的,直接捕获KeyError

#!/usr/bin/env python

from unittest import TestCase

class TestAssertRaises(TestCase):
    def test_assert_raises(self):
        dic = {'one':1, 'two':2}
        with self.assertRaises(KeyError):
            value = dic['three']

执行用例结果

lihui@MacBook  ~  nosetests -sv assertRaises.py
test_assert_raises (assertRaises.TestAssertRaises) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

既然如此,接着看Tempest里这个有关未许可异常的用例

执行之后,报了长篇大论的错误信息

nosetests test_hosts_negative.py:HostsAdminNegativeTestJSON.test_show_host_detail_with_non_admin_user
F
======================================================================
FAIL: tempest.api.compute.admin.test_hosts_negative.HostsAdminNegativeTestJSON.test_show_host_detail_with_non_admin_user
----------------------------------------------------------------------

第一步Token认证,返回200是对的

_StringException: pythonlogging:'': {{{
2017-01-05 23:30:21,509 Request: POST http://10.xx.xx.xx:5000/v2.0/tokens
2017-01-05 23:30:21,509 Request Headers: {'Content-Type': 'application/json'}
2017-01-05 23:30:21,509 Request Body: {"auth": {"tenantName": "Project_qa_admin", "passwordCredentials": {"username": "qa_admin", "password": "xxxxxxxx"}}}
2017-01-05 23:30:21,798 Response Status: 200
2017-01-05 23:30:21,798 Glance request id req-f5e5a51b-ff39-4800-92ff-cddffd6971eb
2017-01-05 23:30:21,798 Response Headers: {'content-length': '4466', 'date': 'Thu, 05 Jan 2017 15:30:20 GMT', 'content-type': 'application/json', 'vary': 'X-Auth-Token', 'connection': 'close'}

第二步获取了host列表,返回200也是对的,注意的是这里执行者还是admin的token

2017-01-05 23:30:21,799 Request: GET http://pubbetaapi.beta.server.163.org:8774/v2/9bd69066f3bd4c9e8c56d4ca7e314330/os-hosts
2017-01-05 23:30:21,799 Request Headers: {'X-Auth-Token': u'a8eb895954ef41b3ab3338e407446430'}
2017-01-05 23:30:22,472 Response Status: 200
2017-01-05 23:30:22,472 Nova request id: req-97f348f1-c78b-4127-bf71-dda3adad49b8
2017-01-05 23:30:22,472 Response Headers: {'content-length': '825', 'content-location': u'http://pubbetaapi.beta.server.163.org:8774/v2/9bd69066f3bd4c9e8c56d4ca7e314330/os-hosts', 'date': 'Thu, 05 Jan 2017 15:30:21 GMT', 'content-type': 'application/json', 'connection': 'close'}
2017-01-05 23:30:22,472 Response Body: {"hosts": [{"zone": "internal", "host_name": "10-10-20-52", "service": "consoleauth"}, {"zone": "internal", "host_name": "10-10-20-52", "service": "scheduler"}, {"zone": "pub.pub1", "host_name": "10-10-10-47", "service": "compute"}, {"zone": "pub.pub2", "host_name": "10-180-0-48", "service": "compute"}, {"zone": "internal", "host_name": "10-180-2-51", "service": "scheduler"}, {"zone": "internal", "host_name": "10-180-2-51", "service": "consoleauth"}, {"zone": "internal", "host_name": "10-10-20-52", "service": "conductor"}, {"zone": "internal", "host_name": "10-180-2-51", "service": "conductor"}, {"zone": "internal", "host_name": "10-10-10-47", "service": "conductor"}, {"zone": "internal", "host_name": "10-180-0-48", "service": "conductor"}, {"zone": "internal", "host_name": "10-180-0-39", "service": "conductor"}]}

第三步又获取了一次token,这次要注意了,是普通租户的token

2017-01-05 23:30:22,473 Request: POST http://10.xx.xx.xx:5000/v2.0/tokens
2017-01-05 23:30:22,473 Request Headers: {'Content-Type': 'application/json'}
2017-01-05 23:30:22,473 Request Body: {"auth": {"tenantName": "Project_bianhuabai@163.com", "passwordCredentials": {"username": "bianhuabai@163.com", "password": “xxxxxxxx"}}}
2017-01-05 23:30:22,595 Response Status: 200
2017-01-05 23:30:22,596 Glance request id req-6a60abbb-267f-4833-b395-cf28be99f87e

最后一步,返回了一个404

2017-01-05 23:30:22,597 Large body (4515) md5 summary: 32e36345598b6d072bfae3313d85a189
2017-01-05 23:30:22,597 Request: GET http://pubbetaapi.beta.server.163.org:8774/v2/10e5051a1cee4f8ebb0e8b5d877de581/os-hosts/10-10-20-52
2017-01-05 23:30:22,597 Request Headers: {'X-Auth-Token': u'1af1546fb6fc414d99435cb91bb24e60'}
2017-01-05 23:30:24,358 Response Status: 404
2017-01-05 23:30:24,359 Nova request id: req-43ba1d9e-5fe8-4ead-b07a-30b991a22a36
2017-01-05 23:30:24,359 Response Headers: {'content-length': '90', 'date': 'Thu, 05 Jan 2017 15:30:22 GMT', 'content-type': 'application/json; charset=UTF-8', 'connection': 'close'}
2017-01-05 23:30:24,359 Response Body: {"itemNotFound": {"message": "Compute host 10-10-20-52 could not be found.", "code": 404}}

整个Test case执行完了,最后还是败了,我们预期的结果也是失败,那么这个test case对还是错?当然错了,预期的是未许可权限失败,而不是此时的404,Compute host 10-10-20-52节点找不到,其实根据这个错误信息,基本不用看整个case的执行流程就知道流程了,首先admin获取了一个环境的host列表,然后用普通租户来获取其中一个host的detail信息,理论上普通租户角色是没权限来查看节点host信息的,因此会出现权限不足

接着就开始找原因了,52是一个控制节点,RESTful API里os-hosts对应了CLI里的的host-describe,因此确认下返回结果正确性

admin来查看控制节点,的确not found

lihui@MacBook  ~/server/source_txt  nova host-describe 10-10-20-52
ERROR: Compute host 10-10-20-52 could not be found. (HTTP 404) (Request-ID: req-995a90c9-14bd-4b46-9026-272df4987054)

admin来查看计算节点,是OK的

✘ lihui@MacBook  ~/server/source_txt  nova host-describe 10-10-10-47
+-------------+----------------------------------+-----+-----------+---------+
| HOST        | PROJECT                          | cpu | memory_mb | disk_gb |
+-------------+----------------------------------+-----+-----------+---------+
| 10-10-10-47 | (total)                          | 28  | 257923    | 1649    |
| 10-10-10-47 | (used_now)                       | 75  | 69120     | 1210    |
| 10-10-10-47 | (used_max)                       | 75  | 60928     | 1200    |
| 10-10-10-47 | 10e5051a1cee4f8ebb0e8b5d877de581 | 1   | 8192      | 20      |
| 10-10-10-47 | 93f5281ad8f54b76b671ec64e662aa2f | 31  | 15872     | 620     |
| 10-10-10-47 | 9bd69066f3bd4c9e8c56d4ca7e314330 | 4   | 4096      | 20      |
| 10-10-10-47 | c7f35d4daa1d4fee840446023a06eb01 | 9   | 8704      | 100     |
| 10-10-10-47 | 7ae30bb692b8482185a1b0381846062a | 4   | 4096      | 40      |
| 10-10-10-47 | 7448dba5ebd14a96b5decd3008a5ab86 | 1   | 1024      | 20      |
| 10-10-10-47 | a17842d3a87a493a9b555bf6aefa9e48 | 13  | 6656      | 260     |
| 10-10-10-47 | f9f27479bccf494ea1597e0ef3a5e8f4 | 12  | 12288     | 120     |
+-------------+----------------------------------+-----+-----------+---------+

普通租户查看控制节点,依旧not found

lihui@MacBook  ~/server/source_txt  nova host-describe 10-10-20-52
ERROR: Compute host 10-10-20-52 could not be found. (HTTP 404) (Request-ID: req-2045114d-43e9-4e78-9539-517fd7a33760)

普通租户查看计算节点,权限不足

lihui@MacBook  ~/server/source_txt  nova host-describe 10-10-10-47
ERROR: User does not have admin privileges (HTTP 403) (Request-ID: req-e3d4c393-40b4-453b-a962-715e442f918f)

也就是我们要测试的其实是通过普通租户来获取“计算节点”的detail信息,不知道为什么偏偏每次选中的都是控制节点,因此看hostname如何获取的

实现代码在下面

    def _get_host_name(self):
        resp, hosts = self.client.list_hosts()
        self.assertEqual(200, resp.status)
        self.assertTrue(len(hosts) >= 1)
        hostname = hosts[0]['host_name']
        return hostname

说实话百思不得其解,获取了host list,然后就把列表的第一个传进去了,如何能保证传的是Compute节点呢?我看一般要么是consoleauth要么是scheduler节点,当然我不清楚XML是啥样的,所以这里就是用例出错的原因,只有计算节点才能真正验证这个test case,所以可以做下过滤,找到计算节点,就退出,传入即可

代码很简单

    def _get_host_name(self):
        resp, hosts = self.client.list_hosts()
        self.assertEqual(200, resp.status)
        self.assertTrue(len(hosts) >= 1)
        #hostname = hosts[0]['host_name']
        for i in range(len(hosts)):
            if hosts[i]['service'] == 'compute':
                hostname = hosts[i]['host_name']
                break
        return hostname

重新执行用例,无误

✘ lihui@MacBook  ~/work/cloud/openstack/tempest-ci/tempest/api/compute/admin   master ●✚  nosetests test_hosts_negative.py:HostsAdminNegativeTestJSON.test_show_host_detail_with_non_admin_user
.
----------------------------------------------------------------------
Ran 1 test in 3.045s

OK

但是要注意的是,如果有其它地方有用到这里的_get_host_name来验证控制节点,就不可行了,需要继续改进

说实话有些怀疑这是Tempest代码么,虽然是H版,写的很多地方莫名其妙,或者有些东西我没GET到

发表回复