在tearDown()方法中是否可以获取测试结果(即所有断言是否已经通过)?我正在运行Selenium脚本,并且希望能够从tearDown()方法内部进行一些报告,但是我不知道是否有可能。
在tearDown()方法中是否可以获取测试结果(即所有断言是否已经通过)?我正在运行Selenium脚本,并且希望能够从tearDown()方法内部进行一些报告,但是我不知道是否有可能。
tearDown()
之前不需要修改任何代码即可正常工作。它可以正确识别修饰符skipIf()
和expectedFailure
。它还与pytest兼容。
代码:
import unittest
class MyTest(unittest.TestCase):
def tearDown(self):
if hasattr(self._outcome, 'errors'):
# Python 3.4 - 3.10 (These two methods have no side effects)
result = self.defaultTestResult()
self._feedErrorsToResult(result, self._outcome.errors)
else:
# Python 3.11+
result = self._outcome.result
ok = all(test != self for test, text in result.errors + result.failures)
# Demo output: (print short info immediately - not important)
if ok:
print('\nOK: %s' % (self.id(),))
for typ, errors in (('ERROR', result.errors), ('FAIL', result.failures)):
for test, text in errors:
if test is self:
# the full traceback is in the variable `text`
msg = [x for x in text.split('\n')[1:]
if not x.startswith(' ')][0]
print("\n\n%s: %s\n %s" % (typ, self.id(), msg))
text
而不是msg
。它只无法识别在expectedFailure块中的意外成功。
示例测试方法:
def test_error(self):
self.assertEqual(1 / 0, 1)
def test_fail(self):
self.assertEqual(2, 1)
def test_success(self):
self.assertEqual(1, 1)
示例输出:
$ python3 -m unittest test
ERROR: q.MyTest.test_error
ZeroDivisionError: division by zero
E
FAIL: q.MyTest.test_fail
AssertionError: 2 != 1
F
OK: q.MyTest.test_success
.
======================================================================
... skipped the usual output from unittest with tracebacks ...
...
Ran 3 tests in 0.001s
FAILED (failures=1, errors=1)
完整代码,包括expectedFailure装饰器示例
编辑:当我将此解决方案更新到Python 3.11时,我删除了与旧Python <3.4相关的所有内容,以及许多次要注释。
if any(error for test, error in self._outcome.errors): ...
。这里有一个更简洁的版本(我没有测试):if any([*zip(*self._outcome.errors)][1]): ...
。 - Mathieu CAROFFexc_list
中,必须进行过滤。如果运行了一个subTest
,则可以通过新版本的Python中的exc_list[-1][0].params.maps
获取其参数。 - hynekcerE AttributeError: 'TestCaseFunction' object has no attribute 'errors'
。 - ebeezerunittest.TestCase.run
的实现,你可以看到所有的测试结果都被收集在result对象中(通常是一个unittest.TestResult
实例),作为参数传递。没有任何测试结果状态留在unittest.TestCase
对象中。
因此,在unittest.TestCase.tearDown
方法中你几乎不能做什么,除非你无情地打破测试用例和测试结果之间优雅的解耦关系,像这样:
import unittest
class MyTest(unittest.TestCase):
currentResult = None # Holds last result object passed to run method
def setUp(self):
pass
def tearDown(self):
ok = self.currentResult.wasSuccessful()
errors = self.currentResult.errors
failures = self.currentResult.failures
print ' All tests passed so far!' if ok else \
' %d errors and %d failures so far' % \
(len(errors), len(failures))
def run(self, result=None):
self.currentResult = result # Remember result for use in tearDown
unittest.TestCase.run(self, result) # call superclass run method
def test_onePlusOneEqualsTwo(self):
self.assertTrue(1 + 1 == 2) # Succeeds
def test_onePlusOneEqualsThree(self):
self.assertTrue(1 + 1 == 3) # Fails
def test_onePlusNoneIsNone(self):
self.assertTrue(1 + None is None) # Raises TypeError
if __name__ == '__main__':
unittest.main()
此方法适用于Python 2.6 - 3.3(已针对新版Python进行修改,请参见此处)。
nosetests
时可能会导致这个问题:https://dev59.com/AWct5IYBdhLWcg3wk-Zq - HugotearDown
和run
结尾处尝试使用print(self.currentResult)
。对于带有F
的测试,failures
计数器在run
中的print
中递增,但似乎在tearDown
中不会递增。这是有意为之吗?我想在tearDown
中知道被“拆除”的单元测试是失败还是成功。 - user3290525unittest.TestCase.run(self, result)
而改用super().run(result)
吗?第一个方法更通用且符合Python风格。 - Premkumar chalmeti注意:我目前无法进行双重检查以下理论,因为离开了开发环境。所以这可能是一次尝试。
也许你可以在 tearDown() 方法内部检查 sys.exc_info()
的返回值,如果它返回 (None, None, None)
,那么说明测试用例成功。否则,你可以使用返回的元组来查询异常对象。
请参阅 sys.exc_info 文档。
另一种更明确的方法是编写一个方法装饰器,可以将其添加到所有需要此特殊处理的测试用例方法中。该装饰器可以拦截断言异常,并根据此修改 self
中的某些状态,使得 tearDown() 方法能够了解情况。
@assertion_tracker
def test_foo(self):
# some test logic
raise SkipTest
。在这种情况下,这可能不是您想要的结果。 - Clandestineexc_info()
这种方法已经失效了,因为在 Python 3 中,只有在最内层的 try: ... except: ...
块中才能访问原始的 sys.exc_info()
,在外部会自动清除。但是,在 Python 3.2 到 3.7-dev 版本中,模块unittest会在离开 "except" 块之前保存 exc_info()
或将其重要部分转换为字符串。(我现在已经验证了这些Python版本上的情况。) - hynekcer如果您使用的是Python 2,可以使用方法_resultForDoCleanups
。此方法返回一个TextTestResult
对象:
<unittest.runner.TextTestResult run=1 errors=0 failures=0>
您可以使用此对象检查测试的结果:
def tearDown(self):
if self._resultForDoCleanups.failures:
...
elif self._resultForDoCleanups.errors:
...
else:
# Success
如果您使用的是Python 3,您可以使用_outcomeForDoCleanups
:def tearDown(self):
if not self._outcomeForDoCleanups.success:
...
._outcomeForDoCleanups
已在3.4中消失。虽然有一个叫做 ._outcome
的东西,但它似乎没有暴露测试通过/失败的状态... - hwjpfailures
属性似乎没有设置,导致 tearDown
抛出 AttributeError
。 - Pieterunittest
的。它不兼容Py.test。 - Pieter这取决于您想要生成什么类型的报告。
如果您想在失败时执行某些操作(例如生成截图),而不是使用tearDown()
,则可以通过覆盖failureException
来实现。
例如:
@property
def failureException(self):
class MyFailureException(AssertionError):
def __init__(self_, *args, **kwargs):
screenshot_dir = 'reports/screenshots'
if not os.path.exists(screenshot_dir):
os.makedirs(screenshot_dir)
self.driver.save_screenshot('{0}/{1}.png'.format(screenshot_dir, self.id()))
return super(MyFailureException, self_).__init__(*args, **kwargs)
MyFailureException.__name__ = AssertionError.__name__
return MyFailureException
failureException
应该接受一个参数(请参见https://docs.python.org/2.7/library/unittest.html#unittest.TestCase.addTypeEqualityFunc)。其次,它被记录为一个“异常”,而你已经用一个函数替换了它。原则上,只要没有其他代码实际上依赖于`failureException`是一个异常类(即`raise self.failureException`现在将在以前成功的地方失败),那么这应该是可以的。 - mgilson继amatellanes的答案之后,如果你使用的是Python 3.4版本,则无法使用_outcomeForDoCleanups
。以下是我设法编写的代码:
def _test_has_failed(self):
for method, error in self._outcome.errors:
if error:
return True
return False
虽然不太好看,但似乎起作用了。
import unittest
import functools
def _tag_error(func):
"""Decorates a unittest test function to add failure information to the TestCase."""
@functools.wraps(func)
def decorator(self, *args, **kwargs):
"""Add failure information to `self` when `func` raises an exception."""
self.test_failed = False
try:
func(self, *args, **kwargs)
except unittest.SkipTest:
raise
except Exception: # pylint: disable=broad-except
self.test_failed = True
raise # re-raise the error with the original traceback.
return decorator
unittest
通过 异常 检测失败的测试。据我所知,唯一需要处理的特殊异常是 unittest.SkipTest
(它并不表示测试失败)。所有其他异常都表示测试失败,因此当它们冒泡到我们这里时,我们将其标记为测试失败。class MyTest(unittest.TestCase):
test_failed = False
def tearDown(self):
super(MyTest, self).tearDown()
print(self.test_failed)
@_tag_error
def test_something(self):
self.fail('Bummer')
每次编写这个装饰器都会变得非常烦人。有没有办法可以简化呢?是的,我们可以编写一个元类来处理应用装饰器:
class _TestFailedMeta(type):
"""Metaclass to decorate test methods to append error information to the TestCase instance."""
def __new__(cls, name, bases, dct):
for name, prop in dct.items():
# assume that TestLoader.testMethodPrefix hasn't been messed with -- otherwise, we're hosed.
if name.startswith('test') and callable(prop):
dct[name] = _tag_error(prop)
return super(_TestFailedMeta, cls).__new__(cls, name, bases, dct)
TestCase
子类,然后就万事大吉了:import six # For python2.x/3.x compatibility
class BaseTestCase(six.with_metaclass(_TestFailedMeta, unittest.TestCase)):
"""Base class for all our other tests.
We don't really need this, but it demonstrates that the
metaclass gets applied to all subclasses too.
"""
class MyTest(BaseTestCase):
def tearDown(self):
super(MyTest, self).tearDown()
print(self.test_failed)
def test_something(self):
self.fail('Bummer')
很可能有一些情况这个程序无法正确处理。例如,它无法正确检测到失败的子测试或预期失败。如果您发现我的程序没有正确处理的情况,请在评论中告诉我,我会进行调查。
*如果没有更简单的方法,我就不会把_tag_error
设为私有函数 ;-)
KeyboardInterrupt
应该使用 except Exception
而不是裸的 except
。我同意,这可能无法与子测试正常工作。它也可能无法与预期失败和其他一些情况正常工作。然而,对于广泛的正常情况,它确实可以稳健地工作。 - mgilsonexpectedFailure
装饰器在您的情况下似乎很难修复。子测试只有在使用expectedFailure
装饰器时才会出现问题,但通过拦截expectedFailure
相对容易解决。键盘中断可以正常工作,但在测试失败后,test_failed
的结果将永远不会被看到。我在编写我的解决方案之前去年尝试了您的方法,但我有时需要使用其他测试装饰器,以组合方式调试它们非常困难。另一方面,在新的Python中验证未记录属性中的数据结构是否相同是微不足道的。... - hynekcerexpectedFailure
是很麻烦的。如果你依赖于实现,那么只需要检查func
和测试用例是否具有真值__unittest_expecting_failure__
属性,并在这种情况下_不_设置失败标志即可。但当然,回答的整个重点是_避免_依赖这些实现细节 :-) - mgilsonunittest
模块的一些私有部分,总体感觉像是解决方法。我强烈建议避免这些方法,因为测试结果和测试用例是分离的,你不应该违反这个原则。addError(test, err)
Called when the test case test raises an unexpected exception. err is a tuple of the form returned by sys.exc_info(): (type, value, traceback).
The default implementation appends a tuple (test, formatted_err) to the instance’s errors attribute, where formatted_err is a formatted traceback derived from err.
addFailure(test, err)
Called when the test case test signals a failure. err is a tuple of the form returned by sys.exc_info(): (type, value, traceback).
The default implementation appends a tuple (test, formatted_err) to the instance’s failures attribute, where formatted_err is a formatted traceback derived from err.
addSuccess(test)
Called when the test case test succeeds.
The default implementation does nothing.
Python 2.7.
在unittest.main()之后,您还可以获得结果。
t = unittest.main(exit=False)
print t.result
suite.addTests(tests)
result = unittest.result.TestResult()
suite.run(result)
print result
受scoffey's answer的启发,我决定把无情推向更高一级,并提出了以下想法。
它适用于纯unittest,也适用于通过nosetests运行时,同时适用于Python版本2.7、3.2、3.3和3.4(我没有专门测试3.0、3.1或3.5,因为我目前没有安装这些版本,但如果我正确地阅读源代码,它也应该适用于3.5):
#! /usr/bin/env python
from __future__ import unicode_literals
import logging
import os
import sys
import unittest
# Log file to see squawks during testing
formatter = logging.Formatter(fmt='%(levelname)-8s %(name)s: %(message)s')
log_file = os.path.splitext(os.path.abspath(__file__))[0] + '.log'
handler = logging.FileHandler(log_file)
handler.setFormatter(formatter)
logging.root.addHandler(handler)
logging.root.setLevel(logging.DEBUG)
log = logging.getLogger(__name__)
PY = tuple(sys.version_info)[:3]
class SmartTestCase(unittest.TestCase):
"""Knows its state (pass/fail/error) by the time its tearDown is called."""
def run(self, result):
# Store the result on the class so tearDown can behave appropriately
self.result = result.result if hasattr(result, 'result') else result
if PY >= (3, 4, 0):
self._feedErrorsToResultEarly = self._feedErrorsToResult
self._feedErrorsToResult = lambda *args, **kwargs: None # no-op
super(SmartTestCase, self).run(result)
@property
def errored(self):
if (3, 0, 0) <= PY < (3, 4, 0):
return bool(self._outcomeForDoCleanups.errors)
return self.id() in [case.id() for case, _ in self.result.errors]
@property
def failed(self):
if (3, 0, 0) <= PY < (3, 4, 0):
return bool(self._outcomeForDoCleanups.failures)
return self.id() in [case.id() for case, _ in self.result.failures]
@property
def passed(self):
return not (self.errored or self.failed)
def tearDown(self):
if PY >= (3, 4, 0):
self._feedErrorsToResultEarly(self.result, self._outcome.errors)
class TestClass(SmartTestCase):
def test_1(self):
self.assertTrue(True)
def test_2(self):
self.assertFalse(True)
def test_3(self):
self.assertFalse(False)
def test_4(self):
self.assertTrue(False)
def test_5(self):
self.assertHerp('Derp')
def tearDown(self):
super(TestClass, self).tearDown()
log.critical('---- RUNNING {} ... -----'.format(self.id()))
if self.errored:
log.critical('----- ERRORED -----')
elif self.failed:
log.critical('----- FAILED -----')
else:
log.critical('----- PASSED -----')
if __name__ == '__main__':
unittest.main()
unittest
运行时:$ ./test.py -v
test_1 (__main__.TestClass) ... ok
test_2 (__main__.TestClass) ... FAIL
test_3 (__main__.TestClass) ... ok
test_4 (__main__.TestClass) ... FAIL
test_5 (__main__.TestClass) ... ERROR
[…]
$ cat ./test.log
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_1 ... -----
CRITICAL __main__: ----- PASSED -----
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_2 ... -----
CRITICAL __main__: ----- FAILED -----
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_3 ... -----
CRITICAL __main__: ----- PASSED -----
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_4 ... -----
CRITICAL __main__: ----- FAILED -----
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_5 ... -----
CRITICAL __main__: ----- ERRORED -----
nosetests
运行时:$ nosetests ./test.py -v
test_1 (test.TestClass) ... ok
test_2 (test.TestClass) ... FAIL
test_3 (test.TestClass) ... ok
test_4 (test.TestClass) ... FAIL
test_5 (test.TestClass) ... ERROR
$ cat ./test.log
CRITICAL test: ---- RUNNING test.TestClass.test_1 ... -----
CRITICAL test: ----- PASSED -----
CRITICAL test: ---- RUNNING test.TestClass.test_2 ... -----
CRITICAL test: ----- FAILED -----
CRITICAL test: ---- RUNNING test.TestClass.test_3 ... -----
CRITICAL test: ----- PASSED -----
CRITICAL test: ---- RUNNING test.TestClass.test_4 ... -----
CRITICAL test: ----- FAILED -----
CRITICAL test: ---- RUNNING test.TestClass.test_5 ... -----
CRITICAL test: ----- ERRORED -----
我从这个开始:
class SmartTestCase(unittest.TestCase):
"""Knows its state (pass/fail/error) by the time its tearDown is called."""
def run(self, result):
# Store the result on the class so tearDown can behave appropriately
self.result = result.result if hasattr(result, 'result') else result
super(SmartTestCase, self).run(result)
@property
def errored(self):
return self.id() in [case.id() for case, _ in self.result.errors]
@property
def failed(self):
return self.id() in [case.id() for case, _ in self.result.failures]
@property
def passed(self):
return not (self.errored or self.failed)
@@ -63,6 +63,12 @@
log.critical('----- FAILED -----')
else:
log.critical('----- PASSED -----')
+ log.warning(
+ 'ERRORS THUS FAR:\n'
+ + '\n'.join(tc.id() for tc, _ in self.result.errors))
+ log.warning(
+ 'FAILURES THUS FAR:\n'
+ + '\n'.join(tc.id() for tc, _ in self.result.failures))
if __name__ == '__main__':
$ python3.3 ./test.py -v
test_1 (__main__.TestClass) ... ok
test_2 (__main__.TestClass) ... FAIL
test_3 (__main__.TestClass) ... ok
test_4 (__main__.TestClass) ... FAIL
test_5 (__main__.TestClass) ... ERROR
[…]
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_1 ... -----
CRITICAL __main__: ----- PASSED -----
WARNING __main__: ERRORS THUS FAR:
WARNING __main__: FAILURES THUS FAR:
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_2 ... -----
CRITICAL __main__: ----- PASSED -----
WARNING __main__: ERRORS THUS FAR:
WARNING __main__: FAILURES THUS FAR:
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_3 ... -----
CRITICAL __main__: ----- PASSED -----
WARNING __main__: ERRORS THUS FAR:
WARNING __main__: FAILURES THUS FAR:
__main__.TestClass.test_2
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_4 ... -----
CRITICAL __main__: ----- PASSED -----
WARNING __main__: ERRORS THUS FAR:
WARNING __main__: FAILURES THUS FAR:
__main__.TestClass.test_2
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_5 ... -----
CRITICAL __main__: ----- PASSED -----
WARNING __main__: ERRORS THUS FAR:
WARNING __main__: FAILURES THUS FAR:
__main__.TestClass.test_2
__main__.TestClass.test_4
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_1 ... -----
CRITICAL __main__: ----- PASSED -----
WARNING __main__: ERRORS THUS FAR:
WARNING __main__: FAILURES THUS FAR:
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_2 ... -----
CRITICAL __main__: ----- FAILED -----
WARNING __main__: ERRORS THUS FAR:
WARNING __main__: FAILURES THUS FAR:
__main__.TestClass.test_2
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_3 ... -----
CRITICAL __main__: ----- PASSED -----
WARNING __main__: ERRORS THUS FAR:
WARNING __main__: FAILURES THUS FAR:
__main__.TestClass.test_2
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_4 ... -----
CRITICAL __main__: ----- FAILED -----
WARNING __main__: ERRORS THUS FAR:
WARNING __main__: FAILURES THUS FAR:
__main__.TestClass.test_2
__main__.TestClass.test_4
CRITICAL __main__: ---- RUNNING __main__.TestClass.test_5 ... -----
CRITICAL __main__: ----- ERRORED -----
WARNING __main__: ERRORS THUS FAR:
__main__.TestClass.test_5
WARNING __main__: FAILURES THUS FAR:
__main__.TestClass.test_2
__main__.TestClass.test_4
result.errors
或result.failures
推断出每个情况下测试的结果。(我认为在拆除测试之后处理测试结果在架构上可能更有意义,但它确实使得根据测试的通过/失败状态遵循不同的测试结束过程的完全有效用例变得更加困难...) 因此,我们不能依赖整体的result
对象,而是可以像其他人已经提到的那样引用_outcomeForDoCleanups
,它包含当前运行测试的结果对象,并具有必要的errors
和failures
属性,我们可以使用这些属性在调用tearDown()
时推断出测试的状态:@@ -3,6 +3,7 @@
from __future__ import unicode_literals
import logging
import os
+import sys
import unittest
@@ -16,6 +17,9 @@
log = logging.getLogger(__name__)
+PY = tuple(sys.version_info)[:3]
+
+
class SmartTestCase(unittest.TestCase):
"""Knows its state (pass/fail/error) by the time its tearDown is called."""
@@ -27,10 +31,14 @@
@property
def errored(self):
+ if PY >= (3, 0, 0):
+ return bool(self._outcomeForDoCleanups.errors)
return self.id() in [case.id() for case, _ in self.result.errors]
@property
def failed(self):
+ if PY >= (3, 0, 0):
+ return bool(self._outcomeForDoCleanups.failures)
return self.id() in [case.id() for case, _ in self.result.failures]
@property
_feedErrorsToResult
。@@ -27,17 +27,20 @@
def run(self, result):
# Store the result on the class so tearDown can behave appropriately
self.result = result.result if hasattr(result, 'result') else result
+ if PY >= (3, 4, 0):
+ self._feedErrorsToResultEarly = self._feedErrorsToResult
+ self._feedErrorsToResult = lambda *args, **kwargs: None # no-op
super(SmartTestCase, self).run(result)
@property
def errored(self):
- if PY >= (3, 0, 0):
+ if (3, 0, 0) <= PY < (3, 4, 0):
return bool(self._outcomeForDoCleanups.errors)
return self.id() in [case.id() for case, _ in self.result.errors]
@property
def failed(self):
- if PY >= (3, 0, 0):
+ if (3, 0, 0) <= PY < (3, 4, 0):
return bool(self._outcomeForDoCleanups.failures)
return self.id() in [case.id() for case, _ in self.result.failures]
@@ -45,6 +48,10 @@
def passed(self):
return not (self.errored or self.failed)
+ def tearDown(self):
+ if PY >= (3, 4, 0):
+ self._feedErrorsToResultEarly(self.result, self._outcome.errors)
+
class TestClass(SmartTestCase):
@@ -64,6 +71,7 @@
self.assertHerp('Derp')
def tearDown(self):
+ super(TestClass, self).tearDown()
log.critical('---- RUNNING {} ... -----'.format(self.id()))
if self.errored:
log.critical('----- ERRORED -----')
当然,前提是这个类的所有消费者都记得在各自的tearDown
方法中调用super(…, self).tearDown()
…
免责声明:本文仅供教育参考,请勿在家中尝试等等。这个解决方案并不是我特别引以为傲的,但它似乎足够好地运行了一段时间,并且是我在一个周六下午折腾了一两个小时后能想到的最好办法…